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FOREWORD 



When the U.S. Office of 
Education was chartered in 
1867, one charge to its 
commissioners was to determine 
the nation's progress in 
education. The National 
Assessment of Educational 
Progress (NAEP) was initiated 
a century later to address, in 
a systematic way, that charge. 

Since 1969, the National 
Assessment has gathered 
information about levels of 
educational achievement across 
the country and reported its 
findings to the nation. It has 
surveyed the attainments of 
9-year-olds , 13-year-olds, 
17-year-olds and sometimes 
adults in art, career and 
occupational development, 
citizenship, literature, 
mathematics, music, reading, 
science, social studies and 
writing. All areas have been 
periodically reassessed in 
order to detect any important 
changes. To date, National 
Assessment has interviewed and 
tested nearly 1,000,000 young 
Americans. 

Learning-area assessments" 
evolve from- a — c-on^ensus- 
process. Each assessment is 
the product of several years 
of work by a great many 
educators, scholars and lay 
persons from all over the 
nation. Initially, these 
people design objectives for 
each subject area, proposing 



general goals they feel 
Americans should be ach ieving 
in the course of their 
education. After ca.reful 
reviews , these objectives are 
given to exercise (item) 
writers, whose task it is to 
create measurement instruments 
appropriate to the objectives. 

When the exercises have passed 
extensive review^ by 
subject-matter special ists , 
mea sur ement experts and lay 
persons, they are administered 
to probability samples. The 
people who compose these 
samples are chosen in such a 
way that the results of their 
assessment can be generalized 
to an entire national 
population.-- That is, on the 
basis of the pe r f o rma nee of 
about 2,500 9-year-olds on a 
given exercise, we can make 
generalizations about the 
probable perfo rmance of all 
9 -year-olds in the nation, 

After assessment data have 
been col lec ted , scored and 
analyzed, the National 
Assessment publishes reports 
and disseminates the results 
_-, as widelyjas . po.ssib.l_e . JNot all- 
exercises are, released for 
publication. Because NAEP 
will readmin ister some of the 
same exercises in the future 
to determine whether the 
performance levels of 
Americans have increased, 
rema ined stable or decreased , 
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it is essential that they not 
be released in order to 



preserve the integrity of the 
study. , ^ 
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INTRODUCTION 



The National Assessment: of 
Educational Progress ''.(NAEP) 
has. completed two assessments 
o£ music, the first conducted 
in 1971-72 and the second dur- 
ing ^1978-79. Each assessment 
surveyed the music achievement 
and" attitudes of American 9-, 
13- and 17-year-olds , using 
deeply stratified, multistage^ 
probability - sample < design. 
This handbook describes the 
procedures i used to administer 
and analyze\the results of the 
1978-79 musdc assessment. It 
also describes -changes in pro- 
cedures between the assess- 
ments. 

To measure changes in perform- 
ance between 1971-72 and 
1978-79, approximately half of 
the exercises in ) the first 
assessment were reassessed in 
the second underf almost iden- 
tical administr 
tions. To measur 
of music achieve 
79, ~ National 
sultants revised 



tive condi- 
the status^ 
lent in 1978-'" 
essment con- 
the objec- 



dev : 



tives used {jm.tlye 
sessment an 
tional exercis 
coverage of the 
tives. 1 



first as- 
d -add i - 
to provide 
revised objec- 



i 1 

Because of cuts in funding, 
exercises assfessrng music 
performance were not- 1 
administered, in the 1978-79 
assessment. 



Approximately *"20,000 9-year- 
olds, 25,000. 13-year-olds and 
22,000 17-year-olds partici- 
pated ' in the 1978-79 music 
assessment . Because National 
Assessment reports results for 
groups of students, not indi- 
viduals, it is • not necessary 
for each student tcx respond to 
every item (exercise). Each, 
respondent completed' only one'' 
item booklet of about 45 min- 
utes in length . .Between 2, A0b ' 
and 2,800 students responded, 
to each bcfcklet. I-n 1978-79 
there were eight exercise 
booklets for 9-year-olds, nine 
booklets for I3-<year-olds and 
eight booklets for 17-year- 
olds. . % , 

The exercises for each assess- 
ment were administered* by a 
professional data collection 
staff to~ minimize the burden 
on participating schools and 
to' maximize uniformity of as- 
sessment conditions. Instruc- 
tions and items were recorded 
on a paced audio tape and 
played back to students to 
reduce the potential effect of 
reading difficulties and to 



National Assessment uses the 
term "exercise" to mean an 
assessment 1 item.. The terms 
"exercise" and "item" are used 
interchangeably in this 
report*. , 



xi 



ensure that all students moved 
through the packages at the 
same speed. 

The majority of the items were 
multiple-choice; a few exer- 
cises, from the- first assess- 
ment that were reassessed In 
1978-79 were open-ended. Each 
item included one or more 
parts. 

Multiple-choice items "* were 
scored by an optical scanning 
machine; open-ended % i terns were 
hand-scored by trained scorers 
using scoring guides developed 
'in the 1971-72 assessment. 
These scoring guides defined 
categories of acceptable and 
unacceptable responses. 

National Assessment reports 
estimated percentages of cor- 
rect responses for single 

items. When a report indicates 
that "85% of the 17-year-olds 
gave a correct response," it 
meons that an estimated 85% of 
the 17-year-olds would have 
given a correct response if 
all the 17-year-olds in 
schools across the country had 
been assessed. In addition to, 
reporting national , results, 
NA5P provides data on the per- 
formance of yarious population . 
subgroups within the national 
population,, defined by sex, 
race/ethnicity, region of the 
country, size and type of? com- 
munity 1 ived in and level of 
parental' edub^tion. For the 
1978-7 9 jnusic assessment , 4 some 
data are available on the ex- 
tent of the music training 
and/or experience , either f or-/ 



mal or informal, of the 9-/ 
13- and 17-year-olds. Na- 
tional Assessment aggregates 
percentages of success ,on var- 
ious sets of items to , provide 
data on changes in performance 
between assessments and on the 
differential performance of 
population subgroups . 

V 

This handbook describes the 
procedures used -to develop, 
administer a,nd » analyze the f 
results of the 1978-79 music 
assessment. Its primary pur- 
pose is to provide detailed 
procedural information for 
people interested in repl i eat- 
ing the assessment or in need 
of more information than is 
prdvided in the' reports con- 
taining assessment data/ The 
eight chapters cover objec- 
tives redevelopment, exercise 
creation, preparation of as- 
sessment booklets, sampling, 
data collection, scoring, data 
analysis and reporting. Each 
chapter ex pi a ins the basic 
procedures used for the 1978- 
79 assessment and contrasts 
these procedures to- those used 
in earlier years (if there 
were changes) . 

Appendicular materials cover 
definitions of reporting 
groups, forms used to gather 
background information about 
students and schools, response 
rates, computation of achieve- 
ment measures and procedures 
for smoothing respondent 
weights. A glossary of Na- 
tional .Assessment * terms is 
provided at the ' end of the 
book . 



v. 
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^CHAPTER 1 

OBJECTIVES REDEVELOPMENT 



The primary goal of the Na- 
tional Assessment of Educa- 
tional Progress (NAEP) is to 
report on the current educa- 
tional status of young Ameri- 
cans and to monitor cny 
changes in achievement over 
time. For each learning area 
" £o"be assessed, NAETP a sRs con- 
sultants to develop objectives 
that define the subject area. 
Since the objectives provide 
guidelines for exercise devel- 
opers, consultants are asked 
to * include examples of the 
knowledge, skills* and atti- 
tudes to be assessed at each 
age level. 

Education in America is a col- 
laborative enterprise involv- 
ing a great many people with 
widely differing philosophies* 
Providing information about 
education ^nationwide would be 
considerably easier if there 
were a consensus about the 
means and ends of American 
education, but the fact is 
that Americans have conflict- 
ing and sometimes con trad ic- 
tory values regarding the 
goals of education and the 
means for achieving those 
\cjoals. 4 To develop an assess- 
ment that is truly national in 
scope and takes into account 
the diversity of curricula, 
values and goals across the 
countLy, National Assessment 
^employs a consensus process 



for developing objectives, 
involving representation of 
many different groups of peo- 
ple. \ ' V 

Several types \ of consultants 
helped develop the 1978-7 9 
music obj ect ivas . Co 1 lege and 
u n Tver s i Yy~ "^sp^nial ists - TrT 
music ensured tk .t the objec- 
tives included important con- 
cepts that the schools should 
be teaching. Educators, in- 
cluding - classroom teachers, 
curriculum supervise rs and 
persons involved in teacher 
education, made sure that the 
objectives included concepts, 
skills and attitudes that tlie 
schools are presently teach- 
ing, as well as those they 
should be teaching • Concerned 
citi2ens, parents and other 
interested lay persons had to 
agree that the objectives were 
important for young people to 
achieve, were free of educa- v 
tional jargon and were not 
biase^ or offensive to any 
g roups . Consul t ants we re rep- 
resentative of the different 
regions of the country a t nd 
minority groups. * They also 
represented a range of experi- 
ence with students of differ- 
ent ^ges and community types; 

The objectives for the 1978-79 
music assessment were devel- 
oped during 1972-73. .(The sec- 
ond ' assessment of ,*nusic was 



13 



originally scheduled to occur 
earlier than 1978-79.) This 
redevelopment was done through 
conferences organized and 
conducted by the National 
Assessment staff. The 
ob j aatLLv,es^x_ed e ye 1 opme n t and 
review" 7 conferences involved 
many music educators and music 
specialists. The objectives 
also were reviewed by a panel 
of lay persons to ensurePthat 
they covered topics generally 
considered to be important and 
that they were not biased. 

The objectives for the second 
nusic assessment were built 
upon The frame wo rk~~of ~th er-ab^ - 
jectives used for the first 
assessment, with more emphasis 
placed on affective, respon- 
siveness. Consultants stressed 
the fact that some of the most 
important goals in music are 
difficult to state in observa- 
ble terms and are difficult to 
measure. One such goal is to 
develop aesthetic sensitivity 
within each student. Another 
is to help the student develop 
his or hrt music potential as 
far as i • *ble during elemen- 
tary a, secondary school 
years. These goals are as im- 
portant for the nonperformer 
as for the performer. 

Following the redevelopment of 
the objectives, another large 
group of - music consultants 
reviewed the objectives and 
assigned weights to define how 
much coverage each objective 
should have at each age. 

Prior to the beginning of de- 
velopment of the * music exer- 
cises, an advisory group of 
music consultants was brought 
together to give direction and 



advice to the NAEP staff on 
furcher refinement of the ob- 
jectives and development of 
the assessment . 

Following is the outline of 

the objectives used for the 

i978-7?T~ "music ~ assessment: 
( Music Objectives, Second As- 
sessment, 1980) . 



1. Value music as an impor- 
tant realm of human ex- 
perience 

A. Be affectively re- 
sponsive to music 

B . Be acquainted wi th 
— music from d i . f -le neat 

nations , cultures , 
periods, genres and 
ethnic groups 

C. Value music in the 
life of the individ- 
ual*, family and com- 
munity 

D. Make " and support 
aesthetic judgments 
about music 

IL. Perform music 

A. Sing (without score) 

B. Play (without score) 

C. Sing or play from a 
wr i tten score 

D. Play or sing a pre- 
viously prepared 
pi ece 

III. Create music 

A. Improvi se 

B. - Represent music sym- 

bolically 

IV. Identi f y the elements 
and expressive controls 
of music 

Identify the ele- 
ments of music 
B. Identify the rela- 
tionships of ele- 



ments in 
composition 
C. Demonstrate 
derstand ing 
variety of 
terms, 
ma r kings 



a given 

an un- 
of a 
musical 
expression 
and con- 



duc t i ng — gest-u-r-es— i n - 

a musical context 

Identify and . classify 
music historically and 
culturally 

A. Identify and de- 
scribe the features 
that characterize a 
variety of folk, 
ethnic, popular and 
art music 



Identify and de- 
scribe the music and 
musical style of the 
various styl ist ic 
periods in Western 
civilization (e.g., 
Medieval, Renais- 
sa nee,- — — Ba to que, 
Classical, Roman- 
tic) . Identify rep- 
resentative compos- 
ers of each period 
Cite examples of 
v/ays in which man 
uti 1 izes music in 
his social and cul- 
tural life 
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CHAPTER 2 



JlEVEIJOUBMEN!LQt.JSXERCiaES _ 



Exercise Development 

Groups of consultants and ad- 
visors weighted the relative 
importance of each objective 
to determine how much testing 
time each should have. Exer- 

C iqpq rp^g^ftgRfid Ln 19 78 -79_ 

were reclassified to match the 
1978-79 objectives so 'chat 
change and status reporting 
would be similar. 

Exercises were developed in 
groups to measure a single 

objective or portion of an 

objective. Every effort was: 
made tc develop groups of ex- 
ercises that could be given to 
all of the age levels, with 
changes in vocabulary, stimuli 
and format when necessary. 

Item development took place 
from February 1977 to January 
1978, with approximately 50 
music specialists and music 
educators participating as 
item writers. Because of re- 
duced funding, which prevented 
the assessment of music per- 
formance, development was di- 
rected to the three remaining 
objectives: Objectives I, IV 
and V. (Development of music 
performance exercises was con- 
ducted during fall' 1978 but, 
as previously mentioned, they 
were' not ' included in the 
1978-79 assessment.) 



There were many exercise writ- 
ing conferences, review con- 
ferences, two field pretest- 
ings and a final selection 
conference. The methods for 
achieving quality exercises 
varied with each conference. 
Typically, each group of musi c 
consultants worked on develop- 
ing exercises for a specific 
objective or for % specific 
age. When a large group met, 
the consultants divided into 
subgroups for this purpose. 
Individuals or small groups 
-.generated. _ items. Then the 
entire group came together to 
review all new i terns • Dur ing 
item review, consultants con- 
sidered age- level appropr iate- 
ness, accuracy of content, how 
well the item measured an ob- 
jective or subobjective and 
readabil i ty . Fol lowing the 
review procedure, more writing 
occurred , f o 1 lowed by more 
reviewing, and so on until the 
group had completed the as- 
signment. Exercises^ that came 
out of the development confer- 
ences were edited by the Na- 
tional Assessment staff to fit 
NAEP format and technical re- 
quirements and were prepared 
for the field "tryouts," or 
pretesting . 

Field Tryouts 
Music exerc ises for the 1978- 
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79 music assessment were field 
tested in schools across the 
country during spring 1977 and 
again in fall 1977. The exer- 
cises were field tested in 
situations as close as possi- 
ble to those of the actual 
assessment to discover poten- 

-tlaJ proble ms in wording, di- 

rections or administrative 
procedures and to collect item 
statistics, timing information 
and scoring information. "Try- 
out" schools were selected to 
represent high- and low-income 
communities as well as more 
typiqal communities. The try- 
outs were conducted with stu- 
dents in at least four class- 
rooms — (-a^p^o-x-ima-fee 1 y 100 stu- 
dents) at each of the ages 
assessed. In order to simu- 
late actual assessment field 
procedures, students recorded 
their answers in the test 
booklets; directions and ques- 
tions were read to students 
from an audio tape; and Na- 
tional Assessment staff mem- 
bers, rather than classroom 
teachers, administered the 
test . 

Completed tryout booklets were 
scored and data were affalyzed. 
Data for the items, as well as 
the administrators' reports of 
any field problems, helped 
both NAEP staff and consult- 
ants evaluate c,nd revise the 



exerc ises . Fo 1 lowi ng the 
spring 1977 field tests, a 
development process like that 
descr ibed above was used to 
produce add i t ional exerc ises 
or to revise or rework exer- 
cises from the first tryouts 
in preparation for fall 1977 
f ield testing . 



Exercise Rev i ews and 
Final Selection 

Each music exercise that had 
been pretested was included in 
a pool of items that was re- 
viewed in a series of confer- 
ences by at least 25 different 
consultants: music educators, 
music specialists, -%c±B£sroon! 
teachers and interested lay 
citizens. Exercises for each 
age group wer^ reviewed for 
appropriateness by teachers 
who taught students at that 
age. Lay citizens, represent- 
ing a variety of occupations 
and interests , also reviewed 
the exercises, checking for 
sex or race bias and consider- 
ing the general importance of 
each exerc ise . 

After the review conferences, 
a final set of exercises for 
the 1978-79 assessment was 
selected by another group of 
music consultants. 
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CHAPTER 3 
PREPARATION OF ASSESSMENT MATERIALS 



words, the answer to one 
e x~e~rcfs e "ucru H - n o t - be con- 
tained in another exercise 
in the same booklet. 



Overview of Packaging 

Following the selection of 
music exercises, National As- 
sessment staff grouped and 
sequenced them into exercise 
booklets. The 1978-79 assess- 
ment was a combined assessment 
of music, art and writing. 
Because of the length of many 
of the art and writing exer- 
cises and because of the many 
handouts used with them, only 
two areas were packaged to- 
gether to avoid administrative 
problems. Booklets included 
'either music and writing exer- 
cises or art and writing exer- 
cises. Since different ages 
received somewhat different 
sets of exercises, there were 
separate booklets f or .,each- acje 
level. Thus, exercises for 
9-year-olds were not sequenced 
in the same order as those for 
13-year-olds, and so forth. 

The following constraints were 
observed in preparing exercise 
booklets: 

1. Each booklet contained 
exercises of varying dif- 
ficulty so that students 
v/o'uld not become bored by 
many easy exerc ises or 
discouraged by many diffi- 
cult exercises. 

2. Exerc ises could not cue 
other exercises. In other 



3* The booklets included 
writing and music exer- 
cises. Each >ooklet was 
turned so that it would 
take no more than 45 min- 

p utes of a student's time 
— the length of a typical 
class period. Booklets 
conta ined approximately 
30-35 minutes of exercise 
t ime and an additional 
10-15 minutes of introduc- 
tory material, instruc- 
tions and background ques- 
tions. The total amount 
of exerci j time devoted 
to music across all pack- 
ages for an age level was 
77 minutes at age 9, 81 
minutes at age 13 and 86 
minutes at age 17. 

4. Booklets were designed to 
be, insofar as possible, 
parallel with respect to 
the number of different 
objectives measured and 
difficulty levels. Items 
measuring a particular 
objective were scattered 
throughout the booklets so 
that many different stu- 
dents would respond to 
questions related to a 
particular objective. 



is 



National Assessment makes 
every effort to minimize dif- 
ficulties connected with the 
testing situation so that re- 
sults will be, as nearly as 
possible, an accurate reflec- 
tion of what students know and 
can do. For example, students 
marked their answers directly 
in the assessment bookle ts , 
not on separate answer sheets. 
It was felt that this proce- 
dure would reduce possibili- 
ties for errors in marking 
answer sheets, especially for 
the younger students. To mini- 
mize guessing, students were 
encouraged to select the "I 
don't know" response option 
included with each multiple- 
choice item or to write "I 
don't know" on the answer line 
for 6pen-ended questions if 
they felt they did * not know 
the answer to a question-. 

Paced audio tapes were pre- 
pared for each exercise book- 
let. Instructions, most of 
the written portions of an 
exercise stimulus^and response 
options were read aloud " to 
minimize the effect of any 
reading difficulties and to 
ensure that all students moved 
through the booklets at the 
same speedc During the field 
testing of the exercises, ad- 
ministrators had determined 
the time needed for most stu- 
dents to respond to an item. 
In addition, the use of tapes 



helped to ensure uniform as- 
* sessment conditions across, the 
country. 



Differences in Packaging: 
1971-72 and 1978-79 

National Assessment makes 
every effort to make assess- 
ment conctl tions f o't i terns 
measur ing change identical 
from assessment to assessment 
so that any changes observed 
will be attributable to 
changes in achievement rather 
than a response to an altered 
testing cond i t ion. 

However, in 1971-72, music and 
social studies were assessed 
together. In 1978-79, music 
and writing were assessed to- 
gether. Because many of the 
writing exeicises were quite 
lengthy, the number of exer- 
cises per package in 1978-79 
was considerably fewer than 
the number of exercises per 
package in 1971-72. 

Some modification occurred in 
the tape recorded read ing of 
the exercises from 1971-72 to 
1978-79. In 1971-72, the "I 
don't know" choice was read 
aloud for each exercise. In 
1978-79, this response choice 
was read aloud only for those 
1971-72 exercises reassessed 
to measure changes in achieve- 
ment. 
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CHAPTER 4 
SAMPLING 



This cha pt er gives an overview 
of the procedures used in de- 
signing and selecting the Na- 
tional Assessment samples for 
the 1971-72 and 1978-79 music 
assessments. Sample design 
and selection for both assess- 
ments were conducted by the 
Research Triangle Institute, 
Raleigh, North Carolina, and 
monitored by National Assess- 
ment staff. 

"The target populations for 
each of the assessments in- 
cluded 9-, .13- and 17-year- 
olds 2 enrolled in either pub- 
lic or private schools at the 
time of the assessment. 
Seventeen-year-olds who either 
left school before graduating 
or graduated early and adults 



For detailed information 
about the 1978-79 National 
Assessment sampling proce- 
dures , see Fi na 1 Report . . .Samp- 
ling and Weighting Activities 
for Assessment Year 10 (1980). 

definition of 1978-79 assess- 
ment age groups are: 9-year- 
olds — born during calendar 
year 1969.; 13-year-olds — 
born during calendar year 
1965; and 17-year-olds ~ born 
October 1, 1961, through Sep- 
tember 30, 1962. 



26-35 years old were also in- 
ciuded in the 1971-72 — assess- 
ment. However, assessment of 
out-of-schoo 1 1 7-year-o Ids and 
adults is quite expensive, and 
in 1978-79 funds to assess 
these - populations were not 
ava i lable • 

Age-eligible persons who were 
f unctional ly handicapped to 
the extent that they could not 
participate in an assessment 
were not considered part of 
the ^target populations. Spe- 
cific groups excluded were: 
non-Engl ish- speaking persons , 
those identified as non- 
readers, persons physically or 
mentally unable to respond and 
persons in institutions or 
attending schools established 
for the physically or mentally 
handicapped. 

National Assessment does not 
follow up specific individuals 
from ona assessment to the 
next. In other words, the stu- 
dents who participated in the 
1978-79 assessment were not 
the same ones who participatd 
in 1971-72. However, in each 
assessment year , participants 
are carefully selected to rep- 
resent each age level. Jor 
example,, Na tional Assessment 
assessed one set of probabil- 
ity samples of 9-year-olds in 
1972 and a totally different 
set of probability samples of 
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9-year-olds in 1979. Thus, 
when we say that 9-year-olds 1 
achievement declined between 
1972 and 1979, we mean that 
students who were 9 years old 
in 1972 correctly answered the 
same questions more often than 
those who were 9 years old in 
1979. 

The National Assessment sam- 
ples were designed to provide 

.appi oxim -crte^ry— — 2, 50Q E^&panr. 

dents per exercise. These num- 
bers allow reporting of data 
for the nation and for the 
subgroups defined in Appendix 
A. The sample design used to 
obtain representative samples 
of the target populations was 
modified somewhat between as- 
sessments. However, these 
minor changes did not affect 
the comparability of the sam- 
ples. An overview of ttfe gen- 
eral sample design approach 

* follows. 



Overview of the National 
Assessment Sample Design 

National Assessment used a 
deeply stratified, three-stage 
national probabil i ty sample 
design with oversampling of 
low-income and rural areas. In 
the first stage, the United 
States was divided into geo- 
graphical units of counties or 
groups of contiguous counties 
meeting a minimum population 
sizexrequi rement . These units, 
callecK primary sampling units 
(PSUs) , x \were stratified by 
region an& N size of community. 
From the list of PSUs, a sam- 
ple of PSUs waV drawn, without 
replacement, with probability 
proportional to N population 
.size measures, representing 



all regions and sizes of com- 
munities. Oversampling of 
low- income and extreme-rural 
areas was first performed at 
this stage by adjusting the 
estimated population size 
measures o f those a reas to 
increase sampling rates. 
Within PSUs, Census Employment 
Survey Data were used to fur- 
ther delineate and oversample 
low-income areas. Counties 
with high proportions of rural 
f ami lies were eftsxr — overs-am 
pled. 

In the second stage, all pub- 
lic and private schools within 
each PSU selected in the first 
stage were listed. Schools 
within each PSU were selected, 
without replacement, with 
probabi 1 i ties propo rtional to 
the numoer of- age-el ig ibles in 
the school • 

The thirjfJ stage of sampling 
occurred during the data, col- 
lection period. A list was 
made of all age-eligible stu- 
dents within each selected 
school. A simple random selec- 
tion of eligible students, 
without replacement,, was ob- 
tained, and item booklets were 
admi nistered to selected stu- 
dents. Specially trained per- 
sonnel selected the sample and 
administered the booklets . 



Survey Weights 

Not all respondents in the 
sample had the same probabil- 
ity of selection because some 
subpopulat ions were over sam- 
pled , and adjustments were 
made to compensate for some 
schools' refusals to partici- 
pate and for student nonfe- 
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sponse. The selection proba- 
bility for each individual was 
computed, and its reciprocal 
was-, used to weight each re- 
sponse in any statistical cal- 
culation to compensate for 
unequal rates of sampling and 
to ensure proper representa- 
tion in the population struc- 
ture. Procedures used to, as- 
sign weights are discussed in 
Chapter 7 and Appendix E. 

-The — mamfear- of PSUs, schools 



within PSUs and students 
within schools was determined 
by optimum sampl ing pr inci- 
ples. That is, a] sample design 
was selected that would 
achieve the maximum precision 
for a g iven level of re- 
sources. Table 1 displays the 
number .of PSUs and' schools 
within PSUs selected in 1971- 
72 and 1978-79 by age. Appen- 
dix C gives information about 
the number of * Students as- 
sessed. 



Age 9 
Age 13 
Age 17 



TABLE 1. Number of PSUs and Schools Within PSUs 
Selected in 1971-72 and 1978-79 



1971-72 
Assessment 
No. of No. of 

PSUs Schools 



116 
116 
116 



777 
890 
780 



1978-79 
Assessment 
No. of No. of 

PSUs Schools 



75 
75 
75 



648 
650 
534 
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CHAPTER 5 
DATA COLLECTION 



Professional data collection 
staff from the Research Trian- 
gle Institute, Raleigh, North 
Carolina, administered the 
assessment booklets. This 
>staff was. used to minimize the 
burden on participating 
schools and to ensure, insofar 
as possible, uniform adminis- 
trative conditions across the 
country ( Final .Report. . . In- 
School^ Field Operations... , 
1979) . 

Participation in the National 
Assessment is voluntary. NAEP 
makes every effort to encour- 
age the schools selected in 
ttte * sample to participate in 
the assessment, and National 
Assessment" and Research Trian- 
gle Institute staffs have ob- 
tained highb rates of school 
cooperation, as shown in Table 
2 ( F'inal Report . . .Field Opera — 



TABLE 2. School Cooperation 
Rates, 1978-79 Assessment 

Age Percent of 

Eligible Schools 
Participating 
in 1978-79 
. 9 Assessment 

9 90.4 

13 90.9 

17 92.9 

Overall 91.3 



tions. . . , 1979, p. 39, Table 
27) . Student cooperation 

rates were also high. The ef- 
fect of student nonresponse is 
discussed in Appendix &r Table 
3 shows the actual number of 
students, that responded to a 
particular exercise booklet at 
each age level in the 1978-79 
assessment . 



In 1978-79, booklets were ad- 
ministered to groups of 10-25 
students; each group respoirded 
to only one of the booklets 
for their age level. The 
groups varied in size depend- 
ing an an estimate of the rate 
of nonresponse for a partic- 
ular school. In 1971-72, the 
planned session sizes were 
fixed at 12 students at each 
age. 

In each assessment, steps were 
taken to guarantee the ano- 
nymity of respondents. Stu- 
dents' names were listed with 
their booklet identification 
number so that scoring and 
processing personnel could go 
back to the school lists for 
data verification — for in- 
stance, on background informa- 
tion — if necessary. These 
lists did .not leave the 
schools and were destroyed six 
months following the assess- 
ment in a school . 
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TABLE 3. Numbei; of Students Responding to Each 
Item Booklet in 1978-79 Assessment, by Age 



An a 


Q 


Ag e 


13 


Ag e 


17 s * 


a aH at 


Ni ittiHp r 


Booklet 

U W W l\ X t 


Nunbe r 


Booklet 


Number 








Re sdo nd i na 




Respond ing 


1 
X 


9 5 ^ 9 


1 


2,755 


1 


2, 730 




9 c; *J 




f \j \j i. 


2+ 


2,746 






-J 


9 77 R 


■? 


? 761 




0 /I Q /I 




9 7Q1 


4 + 


2 772 


5 


2,479 


5 


2,785 


5 


2,684 


6 


2,522 


6 


2,748 


6 


2,739 


7 


2,531 


7 


2,736 


7 


2,642 


8 


2, 524 


8 


2,779 


8 


2,656 


9+ 


2,486 


9 


2,754 


9 


2,787 


10+ 


2,48 3 


10+ 


2,758 


10 


2, 697 


11+ 


2, 526 


11 + 


■ 2,751 


11 + 


2,628 






12+ . 


2,720 


12+ 


2,628 






13+ . 


2,757 


13+ 


2,698 










14 + 


2, 654 


Total 


27,605 




35,910 




37, 822 



+There were no music exercises included in booklets 9, 10 and 11 
at age 9, booklets 10, 11, 12 and 13 at age 13, and booklets 2, 
4, 11, 12, 13 and 14 at age 17. 



To provide information on re- 
spondents 1 backgrounds, school 
officials were asked to re- 
spond to a "pr inci,pal 1 s ques- 
tionnaire." This questionnaire 
asked whether music is taught 
as p required course or as an 
elective course, what type of 
music courses are available to 
students an<* v %he number of 
students Ivfro* participate in 
music courses. Principals of 
9-year-olds K \ schools were 
asked to indicate if music is 
taught and if so, by whom; the 
number of minutes per week 
each student receives music 
instruction; if instrumental 
music instruction is available*- 
and if so, the number of min- 



utes per week that instruction 
is available. Forms used to 
collect background information 
from students and school of- 
ficials are presented in Ap- 
pendix B. 

The assessment administrator 
coded each student 1 s bi rth 
date, sex, grade, racial/eth- 
nic classification and identi- 
fication number on his or her 
booklet. Administrators made a 
visual racial/ethnic identifi- 
cation at, the time each stu- 
dent turned in his or her book- 
let. During the 1978-79 as- 
sessment, six di fferent racial 
classifications were used : 
white, black , Spanish her i- 
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tage, Au .rican Indian or Alas- 
kan native, , Pacific Islander 
or Asian, and unclassified. Ef, 
an administrator was unsure of 
a student's racial/ethnic 
group, he or she referred to 
th.e student's name or listened 
to the student talk °to make 
the identification. Students' 
were not verbally asked by the 
assessment -administrator to 
give a racial identification 
for themselves; however, 17- 
year-old .students were asked 
to provide this information in 
one of the background ques- 
tions included in the exercise 
booklet. 

Sample sizes r o£ the two clas- 
sifications American Indian or 
Alaskan native and Pacific 
Islander ' or Asian are too 
small to- perm it reporting for 
these groups. Results for the 
group classified as Spanish 
heritage cannot be reported 
for separate exercises but can 
JDe reported for aggregate re- 
sults across a number of exer- 
cises. * Change results between 
^71-72 and \L978-79 are re- 
ported for whites, blacks, and 
.Hispanos or Spanish surnamed. 

Each age group was assessed at 
approximately the same time of 
the school year in each as- 
sessment. As noted previously, 
^l^-year-olds were assessed in 
October-December , 9-year-olds 
in January-February and 17- 
year-olds in March-May. 

Following data ' collection, 
assessment administrators s sent 



completed booklets to the 
scoring / contractor, Westing- 
house DataScore Systems, Iowa 
City, / Iowa. Booklets were 
quality-checked to verify that 
correct administrat ive proce- 
dure's/ v/ere being followed by 
the /field staff and. that all 
book/Lets were accounted for. 
Cod^d identification informa- 
tioyh was t also checked for ac- 
curacy; inconsistencies that 
co/Uld not be reconciled were 
sent back to the assessment 
administrator to 'be checked 
against the list^of students' 
names and rdent i f icat ion num- 
bers retained by the school 
for she months following the 
assessment. 



In 1971-72, 17-year-olds who 
t were not currently attending 
school and young adults aged 
26-35 were included in the 
/ assessment. Out-of -school 17- 
year-olds and adul ts could 
each answer up to four book- 
lets cf assessment materials; 
they were paid $5.00 for each 
booklet that they completed. 
Unpaced audio tapes were used 
for these groups. Assessment 
of out-of-school 17-year-olds 
and adults is quite expensive, 
and'in 1978-79 funds to assess 
^these groups were not availa- 
ble. Since out-of-school 17- 
year-olds were not assessed in 
1978-79, only results for 17- 
year-olds attending school in 
1971-72^and 17-year-olds at- 
tending school in 1978-79 were 
^Used in calculating chaYiges in 
17-year-olds 1 performance . 
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CHAPTER 6 S 



SCORING 



Daia collected by National 
Assessment must be converted 
into a form suitable for com- 
puter processing and analysis. 
The conversion was done bye 
Westinghouse DataScore Sys- 
tems, Iowa City, Iowa, the 
contractor responsible for 
printing the assessment exer- 
cise booklets, receiving, 
scoring and machine-processing 
the data. 

While mo$t of the exercises in 
the 1978-79 music assessment 
were multiple-choice, seven 
exercises were open-ended. 
Responses to mul tiple-chroice 
exercises we^e read directly* 
from the booklets by optical 
scanning machines. The scoring 
contractor employed a special 
staff to hand score the open- 
ende^d exercises. Scorers were 
responsible for categorizing 
re/sponses, using the scoring 
guides for open-ended exer- 
cises that defined categories 
of acceptable and unacceptable 
responses. Scorers were also 
responsible for coding the 
category of responses for an* 
exerc^s.e into ovals that could 
be read by optical scanning 
machines. 

To ensure accurate measurement 
of changes * in performance, 
scoring procedures for fypen- 
ended' exercises and correct 
answer keys for multip^- 8 



choice exercises had to be the 
s^me in each assessment. * TKe 
open-ended exercises assessed 
in 1971-72 were reassessed in 
1978-79, and the same scoring 
guides were used in both as- 
sessments However , because 
of the complexity of one exer-* 
cise, the 1971-72 and^l978-79 
responses to this /exercise 
were rescored to ensufe that 
the same procedures were used 
and thus avo id . inconsistency 
across scorers in different 
years. 

Scorers were trained to use' 
the guides by scoring sample 
r9Sponses taken from arriving . 
assessment data. National As- 
sessment staff was involved in 
the training process. Scorers 
initially worked as a g* oup 
and discussed the appropriate 
categorization of each example 
responses Next , scorers worked 
individually on another set of 
responses. Discrepancies were 
resolved and explained . Once 
the . group felt* comfortable 
using the guides, they catego- 
rized the actual* data. Super- 
visory personnel checked all 
work done* for the s first few 
day's of a scoring ' effort to 
ensure /Consistency ; * 

• # ffb further ensure the quality' 
•and consistency of scoring for 
«■ openrended exercises, quali- 
ty-control checks were t con- 



ducted during the scoring of 
these * exercrses. At weekly 
intervals, randomly selected 
responses were drawn from the 
total pool of responses for an 
item and read by randomly se- 
lected scorers. Approximately 
15% of the responses were in- 
cluded in the ^quality-control 
check. Scores for the quali- 
ty-control readings were re- 
corded, and the responses se- 
lected " for quality control 
were then put back into the 
total pool of responses to be 
scored during . the regular 



course of scor ing . Followi ng 
scoring pf'all responses, the 
two scores^ for quality-control 
responses were compared. If 
discrepancies in scoring be- 
came apparent, scorers were 
retrained and, on some occa- 
sions, work was rescored. 



Percentages of agreement be- 
tween quality-control and reg- 
ular scoring were computed for 
each open-ended ' exercise. 
These data are summarized in 
Table 4. 



TABLE 4. Quality-Control Summaries for the 
1978-79 Music Assessment Open-Ended 
Exercises, by Age 



Exercise # 



Age 9 



Age 13 



Age 17 



7 



203001 
203015 
401003 

203001 
203015 
401003 
403001 
404003 

203001 
203015 
401003 
403001 
404003 
40400'' 



% of Scorer 
Agreement 

9.5.^ 
9,9.2 

97.9 i 

95. 1^' 
'-.-97.9 - 
97.5 
100.0 
99.0 

97.6 
98.7 
95.8 
100.0 
. " 100.0 
99.5 



% of Total 
Responses Sampled 

15 
15 

25 • . 

15 
15 
25 
15 
15 

15 ' 
. 15 

15 
' 15 

15 

15 
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CHAPTER 7 



DATA ANALYSIS 



Measures of Achievement 

The basic measure of achieve- 
ment reported by National As- 
sessment is the percentage of 
respondents answering a given 
item acceptably. This per- 
centage is an estimate of the 
percentage of 9-, 13- or 17- 
year-olds who would respond 
acceptably to a given item if 
every 9'-, 13- or 17-year-old 
in the country were assessed. 



mean percentage correct. The 
items included in the Calcula- 
tion of a mean percentage usu- 
ally are located in several 
exercise booklets, and thus 
the mean percentage should not 
be construed - as an average 
test score. 

To present a general picture 
of differences in achievement, 



Percentages of acceptable re- 
sponses are used because each 
ibem is designed as a separate 
measure of some aspect of *an 
objective or subobjective. The 
purpose of National Assessment 
is to discover if more or 
fewer people are able to an- 
swer these items acceptably — 
and thus meet the objectives 
— over time. 

In addition to providing re- 
sults on individual items, 
National Assessment reports 
the average performance across 
groups of similar items — for 
the learning area as a whole, 
for a particular object iveNor 
subobject we , and so forth . 
Such a result constitutes the 
mean, or^ arithmetic, average 
of the estimates of perform- 
ance on the individual items 
of the group and is called the 



^Twenty-two empirical dis- 
tributions of change measures 
from the 1969-70 and 1972-73 
science assessments were used 
to generate Monte Carlo simu- 
lations of sampling distribu- 
tions for several measures of 
central, location. In addition 
to the mean and median, other 
measures of central location 
considered in the simulation 
studies included the average 
of the extremes, two Lorms of 
biweighted estimates and three 
forms 'of weight-matching esti- 
mation described by John W. 
Tukey in the research paper 
"Some Considerations on Loca- 
tors Apt for Some Squeezed- 
Tail (and Stretched Tail) Par- 
ents" (1975). In almost every 
case, the sampling stability 
of the mean change was as good 
as or better than that of the 
other measures studied. 
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National Assessment reports 
the gains- and losses on a 
group of exercises in terms of 
the differences in the average 
percentages of acceptable re- 
sponses. 

Unless the items summarized in 
the mean percentages of 
acceptable responses are iden- 
tical, however, the means of 
one age group should not be ,* 
compa-red directly with the 
means of another, since their 
values reflect both the choice 
of exercises and the perform- 
ance of the students. When 
only a few exercises are 
summarized by a mean, one 
should be especially cautious 
in interpreting results, since 
a small set of exercises might 
not adequately cover the wide 
range of potential behaviors 
included under a given objec- 
tive or subobjective. The 
mean should be interpreted 
literally as the arithmetic 
average of the percentage of 
acceptable responses obtained 
from National Assessment sam- 
ples on a specific set of ex- 
ercises, not as an average 
test score. 

In addition to providing na- 
tional results, National As- 
sessment reports on the 
achievement of various subpop- 
ulations of interest. Groups 
are defined by regiop of the 
country, sex, race/ethnicity, 
size and type of community 
lived in, level of parents' 
education and grade in school 
(see Appendix . A for defini- 
tions of these groups) . 

In considering National As- 
sessment's achievement meas- 
ures, differences in perform- 



ance among ages and between 
assessments are the most use- 
ful. By maintaining the same 
item or set of items in making 
these comparisons, we have a 
reasonable indicator of 

whether more or fewer people 
know or can do something 
'judged important . 

The percentage difference 
between the. performance of any 
one reporting group and that 
of the entire age group 
(nation) is recorded as a 
positive number if the group 
achieved a higher percentage 
or average than the entire age 
group and as a negative number 
if a lower. For example, a 
group performance of +1,8% 
indicates that the percentage 
of responses for the group is 
1.8 percentage points higher 
than the national percentage 
of responses for that age 
level . 

Procedures for estimating 
percentages of responses to 
exercises are dependent on the 
sample design. Each response 
by an individual was weighted 
and multiplied by an 
adjustment factor for 
nonresponse. An estimate of 
the percentages of a 
particular age group that 
would have responded to a 
particular exercise in a 
particular way if the entire 
age group were assessed was 
defined as the weighted number 
of that type of response 
divided, by the weighted number 
of all the responses. A 



^Appendix C discusses nonre- 
sponse in assessment samples. 



29 



similar ratio of weights was 
used to estimate percentages 
and averages for reporting 
groups or subpopulations of 
interest. 3 „ 



Estimating Variability in 
Achievement Measures 

National Assessment used a 
national probability sample at 
each age level to estimate the 
proportion of people who would 
complete an exercise in a 
particular way. The sample 
selected was one of many pos- 
sible samples of the same size 
that^ could have been selected 
using" the same sample design. 
Since an achievement measure 
computed from each of the pos- 
sible samples would differ, 
the standard error of the sta- 
tistic was used as a measure 
of the sampling variability 
among achievement - measures 
from al^ possible samples. A 
standard £££00 based on one 
particular sample, serves to 
estimate; that sampling varia- 
bility. ! 

In the interest of sampling 
and cost efficiencies, Na- 
tional Assessment uses a com- 
plex, stratified, multistage 
probability sample design. 



following the 1978-79 assess- 
ment, a Iweighting-class ad- 
justment procedure was used to 
smooth estimated population 
proportions across the 10 as- 
sessments conducted between 
1969-70 and 1978-79. A discus- 
sion of this procedure is in- 
cluded in Appendix E. 



Typically, complex designs do 
not provide for unbiased or 
simple computation of sampling 
errors. A reasonably good ap- 
proximation of standard error 
estimates of acceptable re- 
sponse percentages and aver- 
ages was obtained by applying 
the jackknife procedure 

(Miller, 1964, pp. 1594-1705; 
Miller, 1968, pp. 567-82; 
Mosteller and Tukey, 1968) to 
f irst-s^tage sampl ing units 
within strata. Standard errors 
for achievement measures such 
as national percentages, group 
differences and means or mean 
differences for a particular 
assessment year were estimated 
directly, taking advantage of 
features of the jackknife pro- 
cedure that are generic to all 
of these statistics. 4 Since 
samples for different assess- 
ments are independent, the 
standard errors of the differ- 
ences in achievement measures 
between assessments can be 
estimated simply by the square 
root of the sum of the squared 
standard errors from ""each of 
the assessments. 

The standard errot provides an 
estimate of sampling reliabil- 
ity for the achievement meas- 
ures used by National Assess- 
ment. It is comprised of sam- 
pling error and other random 
error associated with the as- 
sessment of a specific item or 
set of items. Random error 



^See Appendix D for a more 
detailed description of Na- 
tional Assessment 1 s computa- 
tion of standard errors. 
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includes all possible nonsys- 
tematic error associated with 
administering specific exer- 
cises to specific students in 
specific situations. Random 
differences among scorers for 
open-ended items are also in- 
cluded' in the standard errors. 

National ^Assessment has ad- 
hered ^tjD>-a standard convention 
whereby differences between 
statistics are designated as 
statistically significant at 
the #05 level of significance. 
That is, differences in per- 
formance between assessment 
years -or between a reporting 
group and v he nation are high- 
lighted with asterisks only if 
they are at least twice as 
large as their standard error. 
Differences this large would 
occur by chance in fewer than 
5% of all possible replica- 
tions of our sampling and data 
collection procedures for any 
particular reporting group or 
national estimates . 



Controlling No n random 
Errors 

Systematic errors can be 
introduced at any stage of an 
assessment — exercise devel- 
opment, preparation of exer- 
cise booklets, design or ad- 
ministrative procedures, field 
administration, scoring or 
analysis. These nonsampling, 
nonrandom errors rarely can be 
quantified, nor can the magni- 
tude of the bias they intro- 



duce into the estimates be 
evaluated directly. 

Systematic errors can be con- 
trolled in large part by em- 
ploying uniform administrative 
and scoring procedures and 
requi r ing rigorous qual i ty 
control in all phases of an 
assessment. If the systematic 
errors are the same from age 
to age or group to group, then 
the differences in percentages 
or mean percentages are meas- 
ures with reduced bias because 
subtraction tends to cancel 
the effect of the systematic 
errors. 

Similarly, the effect of^ 
systematic errors in different^ 
assessment years* can , be 
control led by carefully 
replicating in the second 
assessment the procedures, 
carried out in the first. 
Di f f erences in achievement 
across assessment years will, 
also be measures wi th reduced 
bias since subtraction will 
again tend to cancel 
systematic errors . 

Although it is not possible 
for every condition or 
procedure to remain exactly 
the same between assessments 
conducted several years apart, 
National Assessment has made 
every effort to keep 
conditions as nearly the same 
as possible. Changes in 
procedures described in this 
report were judged to have a 
relatively minor impact. 
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CHAPTER 8 
REPORTS ABOUT THE MUSIC ASSESSMENT 



Each assessment generates a 
tremendous amount of data. To 
make these data as useful as 
possible to a variety of audi- 
ences, National Assessment 
provides several types of pub- 
lications and services. 



Reports 

Objectives 

A description of the 1978-79 
music objectives and the pro- 
ced^es used in developing the 
objectives and items for this 
assessment is available in: 
Music Objectives^ Second As- 
sessment • 



Exercise-Level and 
Summary Data 

For those wishing to use spe- 
cific National Assessment 
items, NAEP provides copies of 
released items, exercise docu- 
mentation (including exercise 
timing, objective being meas- 
ured, administrative mode and 
source information " for music 
st'imul i used) and scoring 
guides for open-ended exer- 
cises. Data for the released 
items are provided as addendum 
sheets to the released exer- 
cise set. This loose-leaf set 
of materials is: The Second 
Assessment of Music, 1978-79: 



Released Exercise Set . 

A cassette tape of the music 
stimuli for the released exer- 
cises from the 1978-79 music 
assessment is also ava ilable . 



Overview Report 

In addition to computing exer- 
cise-level data, National As- 
sessment provides means for 
selected sets of exercises 
within a learning area assess- 
ment. These means are useful 
'in comparing performance be- 
tween assessment years and 
between* reporting groups. 
Means are computed for a num- 
ber of reporting variables, 
including region, sex, 
race/ethnicity, size and type 
of community, grade and level 
of parentll education, as well 
as additional variables on the 
kinds of music courses taken 
by 9-, 13- and 17-year-olds. 

Besides publishing many of 
these results in tabular form, 
National Assessment prepares 
overview reports for the gen- 
eral public — including par- 
ents, classroom teachers, 
school administrators and 
legislators — that summarize 
trends and highlight assess- 
ment results. 

The repo rt , M-sic 1971-79: 
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Results From the Second Music 
Assessment , describes the 
1978-79 status in music 
achievement and changes* be- 
tween 1971-72 and 1978-79* 



Public-Use Data Tapes ' 
and User Services 

For those who wish to perform 
their own analyses of National 
Assessment data, computer data 
tapes of respondent-level data 



for the 1978-79 music assess- 
ment will be , made available as 
funding levels permit. 
# 

National " Assessment provides 
some assistance to those wish- 
ing to use the assessment 
items or to replicate assess- 
ment methodology. Those 
Interested in the public-use 
data tapes or in receiving 
assistance should contact the 
National Assessment office. 
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APPENDIX A 

DEFINITIONS OF NATIONAL ASSESSMENT REPORTING GROUPS 



In addition to reporting 
results for all 9-, 13- and 
17-year-old students in the 
United States, National 
Assessment reports results for 
a number of population 
subgroups. Most of these 
subgroups are defined for both 
the 1971-72 and 1978-79 music 
assessments. 



Sex 



Definitions 
follow: 



of the subgroups 



Region 

The country has been divided 
into four regions: Northeast, 
Southeast, Central and West. 
States included in, each region 
are shown on the following 
map. 




Results are reported for males 
and females. 



Race/Ethnicity 

Results are presented for 
blacks, whites and Hispanos. 



Level of Parental Education 

Three categories of parental- 
education levels are defined 
by National Assessment, based 
on students^ reports. These 
categories are: (1) those 
whose parents did not graduate 
from -hiohs school, (2) those 
who have at least one parent 
who graduated from high school 
and (3) those who have at 
least one parent who has had 
some post-high -school educa- 
tion. 



T^pe of Community 

Communities in this category 
are defined by an occupational 
profile of the area served by 
a school as, well as by the 
size of the community in whiclv 
the school is located . This 
category is the only reporting 
category that excludes a large 
number of respondents. About 
two-thi rds do not fall into 
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the classifications listed 
below. Results for the remain- 
ing two-thirds are not re- 
ported k since their performance 
was similar t6 that of the 
nation. 



Advantaged-urban (high-metro) 
communities . Stud-ents in this 
group attend schools in or 
atound cities having a popula- 
tion greater than 200,000 
where a high proportion of the 
residents are in professional 
or managerial positions. 

Disadvantaged-urban (3fow- 

metro) communities . Students 
Yfj this group attend schools 
in or around cities having a 
population greater than 
200,000 where a relatively 
high proportion of the resi- 
dents are on welfare or are 
hot regularly employed. 

* 

Rural communities . Students in 
this group. attend schools i n 
areas with a population under 
"10,000 where many of the resi- 
dents - are farmers or farm 
workers. 



Size of Community 

Big cities . Students in this 
group attend schools within 
the city-limits of cities hav- 
s ing a 1970 census population 
over 200,000. 

Fringed around big cities . 
Students in this group attend 
schools wi thin rrietropol i tan 
.areas (1970 U.S. Bureau of the 
Census urbanized areas) served 
'by cities having a population 
greater than 200,000 but out- 
side the city limits. 



Medium cities . Students in 
this group attend schools in 
cities having a population 
between 25,000 and 200,000, 
not classified in the 
fringes-around-big-cities cat- 
egory. 

Small places . Students in this 
group attend schools in commu- 
nities having 'a population 
less than 25,000, not classi- 
fied in the fringes-around- 
big-cities category. 



Grade in School 

Resul ts are categorized for 
9-year-olds in,, the 3rd or 4th 
grade, 13-year-olds in the 7th 
or 8th grade, and 17-year-ofds 
in the 10th, 11th or .12th 
grade. 



Modal Grade by Region 

Results are categorized for 
9-, 13- and 17-year-old re- 
spondents in grades 4, 8 and 
11, respectively, who live in 
the Northeastern, Southeast- 
ern, Central or Western region 
of the country. 



Modal Grade by 
Community Size 

Resul ts are cat eg or i zed for 
9-, 13- and 17-year-old re- 
spondents in grades 4, 8 and 
11, respectively, who live in 
big cities, fringes around big 
cities, medium cities and 
small places^ 
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Modal Grade by Sex the Y we * e being' taught music 

and 'what kinds of activities 
Results are categorized for they were being exposed to in 
9-, 13- and 17-year-old males their music classes. Thirteen- 
and females in grades 4, 8 and and 17-year-olds were asked 
11, respect ively. about; their music course- 

taking experiences in school. 
All three age groups were 
Music Background asked about their experiences 

Questions with, music outside of school. 

These music background ques- 
Nine-year-olds -were asked if tions are found in Appendix B. 
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APPENDIX B 



FORMS USED TO OBTAIN BACKGROUND INFORMATION 



This appendix includes the 
forms used by National 
Assessment to ^collect 
background information from 
school officials and 



assessment. Following * is a 
listitrg*^ and a brief 
description of the forms 
included* 



respondents .for the 1978-79 



p. 31 

4 

p. 33 
. p. 35 
p. 37 
p. 39 
p. 41 

p. 43 



School Principals Questionnaire — filled 
-ut by school principals or other school 
ff icials for schools at each of the ' age 
levels discussed. . 

Prin cipals Music Questionnaire: Age 9 — 
given to tHe elementary school principals. 
Provides information atfout the elementary 
school* s music program. 

Principal 1 s Mqsic Questionnaire: Age 13 — 
given to the principal at intermediate 
grades. Provides' information about the music 
program in the intermediate grades.^ 

Principals Music Questionnaire: Age 17 ^ — ' 
given to the principal In the senior high 
school. Provides information about the senior* 
high school's music program. 

Stand ard Background Information Form for 
9-year-OrcTs provides information about 

reading material in the home and level of 
parents 1 education. 

Standard Background Information Form for 
13-Year-01ds — provides information about 
reading material in the home, level of 
parents 1 education and place lived in at age 
9. 

Standard Background Information Form for 
17-Vear~01ds — provides information ^ on 
homework, TV watching, racial identification, 
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possessions in the home and classroom 
activities, in addition to questions also 
asked of 9- and 13-year-olds. 

p. 47 Background Information on Music Experiences 
for 9-Year-01ds — provides data on in-school 
muscle .activities of 9-year-olds* 

"^p\ ^50 Background Information on Music Experiences 
^ jfor 13- and 17-Year-01ds — provides data on 
thl^in-school music courses taken by 13- and 
17-year^olds* 

p. 51 Background Information on "Music Experiences 
for 9-, 13- and 17-Year^Olds — provides data 
on the out-of-school music activities .of 
these three age groups* 
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BEST COPY AVAILABLE 



School Principal's Questionnaire 



This rtport « mthomtd by liw (20 UAC \72\ cAh Whitt 
you »ft not rtquirtd to respond, your cooperation rt needed 
to meke th# r«w!u of thit survey comprenenslv*. accurete. 
and timtty 



Primary Sampling Unit L 



.School Number 
Age Group(s) . 



rm 



13 



17 



PLEASE 
PRINT 



Name of School 

Address of School 



(Street) 



(State) 



(Zip Code) 



(City) 

Name of School Principal J 

Name and title of person completing the^form if other than school principal 
Name - Title 



1. What is your best estimate of the current enrollment and the average daily 
attendance by grade of your school (1978-79 school year)? (Enter zeros for 
grades not served by your school.) 



Grade 


K 


1 


2 


3 


A 


S 


6 


7 


8 


9 


10 


11 


12 


Enrollment 




























Average 
Daily 

Attendance 






















1 







Approximately what percentage of the students attending your school live in each 
of the following areas? 

% A In a rural area (less than 2,500) 

% B In a town of 2,500 to 10,000 

% C In a town of 10,000 or more 



100% 



(Items A-C should add to 100%) 
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BEST COPY AVAILABLE 



Approximately what percentage of the students attending your school are ch&dren 

of 

X A Professional or managerial personnel 

X B Sales, clerical, technical or skilled workers" 

X C Factory or other blue collar workers 

X 0 ■ Farm workers 

X E Persons not regularly employed < 

X F Persons on welfare 

■ * 

■ (Items A-F should add to 100%) 



100% 



Approximately what percentage of the students attending your school are** 

X A American Indian or Alaskan Native 

X B Asian or Pacific Islander 

_J X C . Hispanic , regardless of race (Mexican, Puerto RicanJ Cuban, Central 

or South American or other Spanish Culture or or^g^ft) 

X D Black and not Hispanic 

X E White and not Hispanic 

. (Items A-E should add to 100%) 



100% 




Does your school qualify for ESEA Title I assistance? 

Yes - If Yes, approximately what number of ctudents qualify for 

and what number 4 of students are receiving ESEA Title I assistance? 

* 

Approximate number of students qualifying for ESEA 
Title I assistance 

* 

Approximate number of students receiving ES*c/tf Title I 

assistance 
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Principals Music Questionnaire: Age 9 




10. ' Is music taught in your school? 

Cd Yes c=> No (Stop. Do not answer the remaining questions.) 

11. Who does the music instruction in your school? 
<=> The classroom teacher 

<=> A member of a teaching t§am having a background in music but hot 
certified in music, f 

c=> A certified music teacher 

' c=> A combination of classroom teacher and music spf cialist 

c=d Other 

12. Where is music taught in your school? ; 
o In a regular classroom 

a In a special music room 
"CD Other 

13. On the average, how much music instruction time eac^i week do students 
receive? " / 

cz> Less than 30 minutes per week / 



cz> 30 to 59 minutes per week 
cz> 60 to 89 minutes per week 
cd 90 to 119 minutes per week 
120 minutes and over per week 



f 



14. Does your school offer instrumental music lessons for students? 
CD Yes cd No (Go to Question 16) 
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15. On the average, how much instruction time is given each week for 
instrumental music lessons for students? 
cd Less than 30 minutes per week 
cd 30 to 59 minutes per week 

0 

60 to 89 minutes per week i 
90 to 119 minutes per week 

r 

120 minutes and over per week 




16- Do nine-year-old students in your school receive music instruction? 
cd Yes cd No (Stop) * >* 

17. On the average, how much time each week is given to music instruction of 
nine-year-old students? 
cd Less than 30 minutes per week 
cd 30 to 59 minutes per week 
o 60 to 89 minutes per week 
t — > 90 to 119 minutes per week 
120 minutes and over per week 



P.S.U. and 
■ School Number 














( 6 o )( o )( o ) 



CD 



THANK YOU FOR YOUR COOPERATION 



CDCDCDCDCD 
rprprn m CD 
CD CD CEXZDCE) 
CJDC£2CIDC5DCjD 
CDCDCDCJOQD 
CZDCDCZDGDCID 
G3CDCDCE>C3D 
CDCDCDCDCD 



School Name: 



34 

42 



Principal's Music Questionnaire: Age 13 



Is music instruction required for students in each of the following grades in 
your school? 

No 







such 
grade 


Yes 


:no 




.A. 


Grade 6 


CD 


CD 


CD 




B. 


Grade 7 


o 


CD 


CD 




C. 


Grade 8 


cd 


O 


CD 




D. 


Grade 9 


CD 


CD 


CD 






Ungraded : 


CD 


CD 


CD 





(If the answers to ALL parts for Question 14 are NO, go to Question 16 on page 5.) 



If music instruction is required for these grades, for what length of ^ime do 0 
students receive musifc instruction? 

1/4 school 1/2 ' 3/4 Full 
No such Not required^ year or school school school 







grade 


for this grade 


less 


year 


year 


year 


















A— 


Graded 




— — CD" — 


— CD 


CD 


CD : 


— 


B. 


Grade 7 


CD 


CD 


CD 


CD 


CD 


CD 


C. 


Grade 8 


CD 


CD 


CZD 


t 

CD 


CD 


CD 


D. 


Grade 9 


CD 


CD 


'CD 




CD 


CD 


E. 


Ungraded 


CD 


CD 


CD ' 


CD 


CZD 


:cd 
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16. Are elective music courses available for students in each of the following 
grades? 







No such 
grade 


i es 


IN 0 


A. 


Grade 6 


CD 


cz> 




B. 


Grade 7 : 


CD 


CZ) 


cd 


C. 


Grade 8 


CD 


CZ) 




D. 


Grade 9 


CD 


CZ) 


cd 


E. 


Ungraded 


cz> 


CZ) 


CZ) 



(If the answers to ALL parts for Question 16 are NO, go to Question 18 on page 6.) 



r 

17. If music is an elective , what percentage of the students in the following 
grades elect to take music courses each year? 





No 
such 
grade 


Not 
elective 
for this 
grade 


*°- 
°24% 


25- 
49% 


50- 
74% 


75; 

89% 


90- 
100% 


A. 


Grade 6 cd 


CZ) 


CZ) 


CZ) 


CD 


\ CD 


CD 


B. 


Grade 7 cd 


a 


a 


CD 


CD 




CD 


r. 




i — \ 


t — > 


< — > 


_ CZL, 


CD 


CD 




D. 


Grade 9 czd 


a 


a 


CD 


CD 

jS> 


CD 


CD 


E. 


Ungraded cd 


CZ) 


a 


CD 


CD 


CD 


CD 



> 
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Principal's Music Questionnaire: Age 17 



.7. Is music instruction, including music classes, group or private instruction, 
and performing groups, available in your school for the following grades? 

No such 

Grade Yes No 



A. 


Grade 9 (ZD czd 


(ZD 






B. 


Grade 10 czd czd 


(ZD 






C. 


Grade 11 ' czd cz> 


(ZD 






D. 


Grade 12 . o czd V 


(ZD 






E. 


Ungraded czd czd 


(ZD 






Does your school offer the following types of music instruction at some time 


during Grades 9, 10, 11, or 12? 




Yes 


No 


A. 


General Music class or Music Appreciation 




(ZD 


■ cz> 


B. 


Music Literature, Music History or 
Introduction to Music 




(ZD 


cz> 


C. 


Music Theory class or Music Composition 




(ZD 


(ZD 


D. 


Choir, Chorus, Glee Club or other vocal 
music group 




(ZD 


(ZD 


E. 


Vocal lessons (class or private) 




— c~^) — 


CZD 


F. 


Band or Instrumental Music 




(ZD 


(ZD 


G. 


Stage or Jazz Band 




(ZD 


(ZD 


H. 


Jazz class or Improvisation class 




(ZD 


CD 


I. 


Orchestra 




(ZD 


CZ) 


J. 

— 


Instrumental lessons (class or private) 




(ZD 


(ZD 
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p. About what percent of the 17-year-old students are currently enrolled in 
music classes.(n~ot including band, orchestra or vocal groups)? 
cz> 0to24% , ' 

czd 25 to 49% 
c=> 50 to 74% 
czd 75 to 89% 
czd 90 to 100% 
_\ 

10. About what percentage of the 17-year-old students arc currently enrolled in 
performing groups, such as band* orchestra, or vocal music groups (choir, 
chorus, or glee clubs)? 
(ZD 0to24% 
<zd 25 to 49% 
cz> 50 to 74% ' 
(=d 75 to 89% 
cz> 90 to 100% 



THANK YOU FOR YOUR COO PERATION 
School Name: a 



P.S.U. and 
School Number 



mrn m m m 
CIDCIDCIDCIDCID 

( 4 )C~*~~)( 4 3C 4 If 4 

CDCDCDGDCD 
CO CD CO CO CO 



CO CO CO CO CO 
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Standard Background Information Form for 9-Year-01ds 








i. 


Does your family get a newspaper regularly? 
czd Yes o No o I don't know. 








2. 


Does your family get any magazines regularly? 
(ZD Yes o No o I don't know. 








3. 


Are there more than 25 books in your home? 
CD Yes o No o I don't know. , 






J 


4. 

5. 


Is there an encyclopedia in your home? 
CD Yes CD' No CD I don't know. 

How much school did your father complete? ^ 
/FILL, IN THE ONE OVAL which best shows how much school your 
father completed.) 

CD Did not complete the 8th grade 

CD Completed the 8th grade, but did not go to high school 

r— ^ Went t" *"£h school, but did not graduate from high school 




. . — 






CD Graduated from high school 










CD Some education after graduation from high school 










CD I don't know. 








6. 


Did your father graduate from a college or university? 
t — > Yps cd No CD I don't know. 
















- 
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< 


3 * * 


- 


4 



How much school did your mother complete? 

(FILL IN THE ONE OVAL which best shows how much school your 

mother completed.) 

CD Did not complete the 8th grade 

"V. 

CD Completed the 8th grade, But did not go tisUhigh r school 



CD Went to high school, but did not graduate from high school 

( — > Graduated from high schooi 

CD Some education after graduation from high school 

CD I don't know. 

Did your mother graduate from a college or university? 
CD Yes CD No CD I don't know. 
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Standard Background Information Form for 13-Year-01ds 



1. Does your family get a newspaper regularly? 
cz> Yes CD No cz> I don't know. 

2. Does your family get any magazines regularly? 
cz> Yes cz> No cz> I don't know. 

3. Are there more than 25 books in your home? 
t — > Yes cz> No cz> I don't know. 

4. Is there an encyclopedia in your home? 
cb Yes cz> No cz> I don't know. 

5. How much^school did your father complete? 

(FILL IN THE ONE OVAL which best shows hbw much school your 

father completed*) 

CZ> Did not complete the 8th grade 

O, Completed the 8th grade, but did not go to high school 

CZ> Went to high school, but did not graduate from high school 

cz> Graduated from high school 

cz> Some education after graduation from high school 



cz> I don't know. 

6. Did your father graduate from a college or university? 
cz> Yes cz> No cz> I don't know, 

7. How much school did your mother complete? 

(FILL IN THE ONE OVAL which best shows how mubh school your 

mother completed.) 

CZ> Did not complete the 8th grade 

CZ> Completed the -8th grade, but did not go to high school 

CZ> Went to high school, but did not graduate from high school 

CZ> Graduated from high school 

CZ> Some education after graduation from high school 

CZ> I don't know, 

8. Did your mother graduate from a college or university? 
t — > Yes ( — > No cz> I don't know. 



41 



49 



\ 



9. Where did you live on yourjjinth bhthday? 

(ZD In the United States (Please specify the state or territory.) 



(ZD 



Outside the United States (Please specify the country.) 



(ZD I don't know. 
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Standard Background Information Form for 17-Year-01ds 



1. 1 Which of the following does your family have at home? (Fill in one oval on 





■ 


Have 


Do not have 


A. 


Newspaper received regularly 


cd 


cz> 


B. 


Magazines received regularly 


CD 


CZ) 


C. 


1 More than 25 books 


CD 


CZ) 


D. 


Encyclopedia 


CZ) 


CZ) 


E. 


^ Dictionary 


CZ) 


CZ) 




Record player 


CZ) 


CZ) 


G. 


Tape recorder or cassette player 


CZ) 


CZ) - 


\ H - 


Typewriter < 


CD 


CZD 


I, 

i 


Vacuum cleaner 


CZ) 


CD 


J. 


, Electric dishwasher 


CD 


CD 


K. 


Two or more cars or trucks that run 


CD 


CD 



2. How much time did you spend on homework yesterday? 



cd No homework was assigned 

cd- I had homework but didn't do it 

czd Less than one hour 

cd Between i and £ hours J 

cd More than 2 hours ' * 

3. How many different schools have yau attended since you started the fi^st 
grade? * 

cd 1 to 3 schools j 

cd 4 to 6 schools 

cd 7 to 1 9 schools 

cd 10 or more schools . 

/ 
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How long have you lived in the community in which you now live? 



-All my life 

10 or more years but not all my life 
5 to 9 years 
2 to 4 years 
1 year 
cd Less than 1 year 

How much television did you watch yesterday? 



None 

1 hour or less 
1 hour 



2 hours 

3 hours 
Oours 



5 hours 

6 hours or more 



Is English the language spoken most often in your hohie? 
Yes cd No 



Is a language other than English spoken in your hom£? 

o Often cd Sometimes o Never 

How many brothers or sisters do you have who are older than you? 
None 1 2 3 4 5 ; 6 or more 



How many brothers or sisters do you have who are younger than you? 

4 5 6 or more 



None 



12 3 

CD CD 'CD 



A. What is-yotrr racial background? 

American Indian or Alaskan Native 
Asian or Pacific Islander 
cd Black 
o White 

* CD Other (Please specify) 



-ft — H 



*er4tage4iispani< 



m f Puert o Rican, 



Cuban, Central or-South-Americah or other Sp&MshjMtyre or origin)? 
Yes cd No i 
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1 1. How often has each of the following been used in the courses you are taking 
this vear? ('Fill in one oval on each line.) 

Fairly 

Never Seldom Often Frequen 



A. 


listening , to. the teacher's lecture 


» 

CZD 


CZD 


CZD 


CZD 


B. 


Participating in student-centered 
discussions 


o 


. CD 


(ZD 


CD . 


C. 


Working on a project or in a 
.laboratory 


CD 


_ CZD 


CZD 


CD 


D. 


> ■ ' 

Writing essays, themes, poetry,^ 
stories ^ 


CZD 


— - t 
CZD 


CD 


CD 


Eh 


Going 9n field trips 


CZD 


CZD 


CZD 


CD 


F. 


Having individualized instruction 
(small groups or one-to-one with 
o tpnphpr^ 


CZD 


CZD 


CZD 


CD 


. G. 

< 


Using teaching machines or computer- 
assisted instruction 


CZD 


, -CZD 


CZD 


CD 


H. 


Watching television lectures 


CZD 


CZD 


CZD 


CD 

\ « 


I. 


•Studying fronvtextbooks 


CZD 


CZD 


CZD 


\ 


J. 


Library or media center assignments 


CZD 


CZD 


CZD 


CD 



ERJ.C 



A How much school did your father complete? . 
; (FILL IN THE ONE OVAL which best shows how much school your 
father completed.) 
CD Did not complete the 8th grade 
CD, Completed the 8th grade, but did not go to high school 
CD Went to high school, but did not graduate from high school 
CD Graduated from high school 

cd Some education .after gradaatiT5irfrom-hTgh-school 

CD I don't know'. 
B. Did your father graduate from a college or university? 
CD Yes CD No cd I don't know. 
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: - ; * - . . - 




sg!?- -:- * 


c. 


V >. 

\ 

How much school did your mother complete?' 

(FILL IN THE ONE OVAL which best shows how much school your 

mother completed.) v 

cb Did not complete the 8th grade 

cd Completed the 8th grade, but did not go to high school 

cd Went to high school, x but did not graduate from high school 

cd Graduated from high school 

cd Some education after graduation from high school 






D. 
E. 


cd I don't know. 

N 

Did Vour mother graduate from, a college or university? 

cd Yes * cd No cd I don't know. 
Where did you live on your ninth birthday? 

cd In the United States (PleaSste specify the state or territory.) 


__ 






czd Outside of the United States (Please specify the country.) - 








cd I don't know. 






F. 


Where did /you live on your thirteenth birthday? 








cd In the United States (Please specify the state or territory.) 








cd Outside of the United States (Please specify the country.) 

/ 


_- »■ 




• 


czd 1 don't know. • 55 




> 




** 

4 ' ' 

i 
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Background Information on Musiic Experiences, 
for 9-Year-0lds 



Are you being taught music in school this year ? 

cd Yes - v 

o 'No 

czd I don't know. v 

/ 

Were you taught music in school last year ? 

Yes 
No 

I don't know. 



J7 
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Do you ever do each of the following activities in your school music 
class? Fill in one oval in each box on this and the next page. 



In your school music class, do you 



I. 


listen to music? 


Yes 

(ZZ> 


No 

CZD 


I don't know. 

(ZD 


2. 


sing just for fun? 


Yes ' 

CZD 


No 

(ZD 


I don't know. 

> CZD 


3. 


sin£ in a special music 
group such as a choir, 
chorus or glee club? 


Yes 

CLD 


No 

(ZZ) 


I don't know. 
cz> 


4. 


play a musical instrument 
just for fun? > 


Yes 

(ZD 


No 

(ZZ) 


I don't know. 

(ZZ) 



I 



5. play a musical instrument 
in aspecial music group 
such; as a band or orchestra? 

\ ■ ; 


Yes 

(ZZ) 


No 

(ZZ) 


I don't know. 

(ZZ) 


6. read a^out music or 

musicians? 

\ 


Yes 

(ZZ) 


No 

(ZZ) 


I don't know. 

(ZZ) 


\; 

7., make upiyour own music? 


Yes 

(ZZ) 


No 

(ZZ) 


I don't know. 

(ZZ) 



1 

\ 



i 



I 
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"Which one of the following things would you rather do if you had one 
freefperiod a day in school? Choose only one. 

\; 
I 

k 

r 

czd Play a musical instrument 

czd Draw or paint 

czd Write a story 

czd 'Sing in a musical group 

czd Learn a foreign language 

czd Listen U music 

czd None of these 

czd I don't know. 
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Background Information on Music Experiences 
for 13- and . 7-Year-01ds 



Listed below are several kinds of music activities. Indicate how many years 
you have .participated in *ach activity in school. Be sure to fill in one oval in 
each box. 



Including elementary and junior high school, how many years have you taken 
or participated in 



A. 


General Music Class or Music Appreciation? 








Have Never Less Than 
Takea* 1 Year 


1 to2 
Years 


3to4 
Years - 


5 or 
More Years 




CD CD 


CD 


CD 


C3 


B. 


Choir, Chorus or Glee Club? 










Have Never Less Than 
Taken 1 Year 


1 to2 
Years 


3 to4 
Years 


5 or 
More Years 




CD .<=> 


CD 


CD 


CD 


C. 


Band or Instrumental Music? 










Have Never Less Than 
Taken 1 Year 


1 to2 
Years 


% to4 
Years 


5 or 
More Years 




CD CD 


CD 


CD 


CD 


D. 


Orchestra? 










Have Never Less Than 
Taken 1 Year 


1 to2 
Years 


3 to4 
Years 


5 or 
More Years 




CD CD 


CD 


CD 


CD 


E. 


Introduction to Music, Music Literature or Music History? 




Have Never Less Than 
Taken 1 Year 


1 to2 
Years 


3to4 
Years 


5 or 
More Years 




CD CD 


CD 


CD 


CD 


F. 


Music Theory Class or Music Composition? 








Have Never Less Than 
Taken 1 Year 


1 to2 
Years 


3 to 4 
Years 


5 or 
More Years 




CD CD 


CD 


CD 


CD 



\ 

\ 

\ 
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Background Information on Music Experiences 
for 9-, 13- and 17-Year-Olds 



Which of the following activities do you do outside of school ? Fill in one oval 
in each box. 

Outside of school, do you 







Yes No 


I don't know. 




liQton tn mimic ^ 


CZ) CZ) 


CZ) 


B. 


sing just for fun? 


CZ) o 


CZ) 


> 

C. 


sing with friends just for fun? 


o o 


CZ) 


D. 


sing in a church or community 
music group? 


CZ) o 


CZ) 


E. 


play a musical instrument by 
yourself just for fun? 


o o 


CZ) 


F. 


play a musical instrument with 
friends just for fun? 


CZ) CZD 


CZ) 


G. 


play a musical instrument in a 
church or community music group? 

i 


CZ) CZ) 


CZ) 


H. 


take music lessons? 


CZ) CZ) 


CZ) 


I. 


make up your own music? 


CZ) CZ) 


CZ) > 



ERIC 
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APPENDIX C 



RESPONSE RATES FOR ASSESSMENT SAMPLES 



Table C-l shows the response 
rates for students assessed in 
1971-72 and 1978-79. In the 
1971-72 assessment, for each 
of the three age groups, 12 
students and 4 alternates were 
selected for each assessment 
session. If all 1-2 students 
appeared for the session, then 
the alternates were dismissed. 
Otherwise, enough alternates 
were selected to bring the 
size of the group to as many 
student? as possible ; up to a 



total o£ 12. If the group as- 
sessed numbered between 8 and 
12 students", then the adminisr 
tration was considered / com- 
plete. If the final total was 
not at least a quorum of 8, a 
second , and sometimes* a third 
make-up session was held. The 
' percentages in Table C-l are 
- based on the numbers of stu- 
r dents assessed from the origi- 
nal groups of 12 selected and 
do not reflect the use of al- 
ternates. 



TABLE C-l. Number of Students' Assessed and Percent of 
Sample Covered", by Age and Assessment Year 



Year 



1971-72 









Total 


Average 


Average 




Type 




Number 


Number 


Sample 




of : 


Number 


of 


Assessed 


Coverage 




Adminis- 


of 


Students 


Per .* 


in 


e 


tration 


Package's 


Assessed 


Package 


Percent 


9 


G 


• 4 


10,824 


2,706 


89.2 




I 




6,953 


2, 318 


90.0 


13 


G 


7 


18,669 


2,667 


87.0 


I " 


3. 


' 6,870 


2, 290 


88.5 


17 


G 


9 


21',233 


2,359 


78.2 


I 


3 


6,565 ■ 


2,188 


84.1 


9 


G 


< ii 


■ 27,605 


2,510 


87.8 


13 


G 


13 ' 


35,910 


2,762 


. 84.9 


17 


G 


14 


37,822 


2,702 


77.8 



1978-79 



+In the 1971-72 assessment, some booklets were administered to 
individual (I) students using*" an interview mode. The other 
booklets were administered to groups (G) of students. 

\ 



For the 1978-79 assessment, 
slightly different procedures 
were used. The number of stu- 
dents selected for each admin- 
istrative session varied from 
16-25 students depending on 
previous response rates ob- 
tained from schools in similar 
communities. No alternates 
were selected. The quorum size 
needed to consider an adminis- 
trative session complete var- 
ied according to the number of 
students selected. Since non- 
response rates have always 
been relatively small for ages 
9 and 13, the make-up or fol- 
low-up procedures used in 
1978-79 for these ages were 
similar to, the ones used for 
the first mi ic assessment. If 
a "quorum wa. not obtained at 
the first administrative ses- 
sion, a second and sometimes a 
third make-up session , was 
held. At age 17, in the 1978- 
79 assessment, follow-up pro- 
cedures were conducted on a 
school, rather than a session, 
•basis. If a school had an 
overall response rate of less 
than 75%, then all nonrespon- 
dents in the school were con- 
tacted for one or two follow- 
up sessions. These follow-up 
procedures for 17-year-olds 
provided sample coverage simi- 
lar to that obtained at ages 9 
and 13. 

Since response rates at age 17 
have always been somewhat 
lower than at the other two 
ages, the Research Triangle 
Institute (RTI) , Raleigh, 
North Carolina, was asked to 
conduct a special study of 
nonrespondents during the 



1972-73 assessment of science 
and - mathematics. The results 
(Kalsbeek et al., 1975; Rogers 
et al*, 1977) indicated that 
about 80% of the total nonre- 
spondent- group did not appear 
at the assessment sessions 
bpcause of conflicting school 
activities or illness. The 
remaining 20% did not seem to 
be ava ilable. They attended 
school infrequently, if at all 
( f o r practical purposes > they 
had dropped out), or they had 
moved out of the school atten- 
dance area. In either case, 
these students probably should 
not have been listed in the 
in-school ' population gt 
el igibles. 

Tables published ' in previous 
National Assessment repo rts 
showing response rates for age 
17 generally contain percent- 
ages adjusted to account for 
those 17-year-olds listed, but 
not attending school. But, 
since National Assessment has 
not m had the resources to rep- 
licate the RTI study in recent 
assessments, the 20% figure 
used as a basis for adjusting 
these percentages may be out- 
dated and thus, the percent- 
ages given in Table C-l have 
not been adjusted. It seems 
likely that despite efforts to 
update the lists of eligibles, 
these lists still contain some 
percentage of students ' who 
have in effect left the 
schools. Thus, the percentages 
listed for age 17 are probably 
underestimates of the actual 
response rates for 1 7-year- 
olds attending school. 
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APPENDIX D 

COMPUTATION OP MEASURES OF ACHIEVEMENT, 
CHANGES IN ACHIEVEMENT AND STANDARD ERRORS 



Several measures of achievement that 
National Assessment us^o in its re- 
ports are^ described in- Chapter 7 of 
this docunent. The sample design, as 
described in Chapter 4, is a complex, 
deeply stratified, multistage proba- 
bility sample design. Measures of 
achievement are Obtained by weighting- 
individual responses appropriately. 
Reasonably good approximation of 
standard error estimates of these 
achievement measures can be obtained 
by applying the jackknife procedure 
to first-stage sampling units within 
strata^ using' the method of succes- 
sive differences and accumulating 
across strata. 

In this section, the measures of 
achievement are first defined^ln al^ 
gebrajc form, followed by a descrip- 
tion of the jackknife method that- 
National Assessment us€s to estimate 
* the standard errors of achievement 
measures. 



Measures of Achievement 

Based on the • sample design, a weight 
is assigned to every individual who 
responds to an exercise administered 
in an assessment. Hie weight is the 
reciprocal of the probability of se- 
lecting a particular individual to 
take a particular exercise with ad- 
justment for nonresponse. Since the 
probabilities of selection are based 
on an estimated number of people in 



the target age population, the weight 
for an individual estimates the num- 
ber of similar people that individual 
represents in the age population. As 
explained in Appendix E, the weights 
are adjusted to reflect information 
from previous assessments on popula- 
tion distributions. 

A sum of the weights for all individ- 
uals at ah age level responding to an 
exercise is an estimate of the total 
number of people in that age popula- 
tion. A sum of weights for all indi- 
viduals at an age responding cor- 
rectly to an exercise is an estimate 
of the number of people in the age 
population 'who. would be able to re- 
spond correctly if the entire popula-^ 
t ion— were assessed — These - concepts^ 
also apply to , any reporting group 
(e.g., defined by region, sex, and so 
on) and category of response (e.g r , 
correct, incorrect and " I don' t 
know"). 



Let W? u . 
■ lhk 



sum of weights for respon- 
dents to exercise e who 
are in reporting subgroup 
i_ and' who are in the j<th 
replicate of the hth sam^ 
pling stratum, and 



C9^ k = sum of weights for respon- 
dents to exercise e who 
are in reporting subgroup 
j,, who are in the kth rep- 
licate of the hth sampling 
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stratum and who selected 
response category 2 ( e ^»/ 
correct foil) for the ex- 
ercise. 



portion in subgroup who would" se- 
lect category 2 on exercise e ancl the 
proportion in the nation is denoted 

by: • . 



Note that Wf hk = I C^ R . 

Then summing k over the sample 

repl icates in the stratum h , and sum- 
ming over the H sampling strata, 

— H n, 

h=l k=l 

estimates the number of eligibles in 
the population who are in subgroup i* 



^3 



(?) AP 



H n 

Similarly, cf^ = E > E h cf^ k esti- 

li = l k = l 

mates the number of eligibles in the 
population who are in subgroup _i and 
who would select response category 2 
*for exercise e. 

An estimate of the proportion of. the 
eligibles in the age population in 
group ^ who would select response 
category 2 on exercise ie is: 



National Assessment ale . reports the 
arithmetic mean of the percentage of 
correct responses over sets of exer- 
cises corresponding to. the measures 
in (1)," (2) and (3). Ihese means are 
taken over the set of all exercise 
or a subset of exercises classified 
by a reporting topic or content ob- 
jective. The mean percentages of 
correct responses taken over m exer- 
cises in some set of exercises corre- 
sponding to measures (1), (2) and (3) 
are, respectively: 



(A) P, 



i 

m 1 
e 



c e /W e 



e 



(6) » ± + P t - P A . 



(1) P 



c ej /w e 



In the special case where the pro*- 
portion of all age-eligibles who 
would select response category 2 on 
exercise e is estimated, the index A 
.(for ALL) will be used in place of i. 
as follows: 



(2) p' 



ej 



c eJ /W e . 
A++ A++ 



In National Assessment reports, the 
proportion in (1) multiplied by 100 
is called the group percentage, and 
th?r proportion in (2) multiplied by 
100 is called the national percent- 
age. The difference between the pro- 



Note that the response category sub- 
script. 2 has been suppressed since 
the means are understood to be taken 
over the correct, response category 
•for each exercise. 

Each of these six achievement meas- 
ures is computed and routinely used 
in reports describing achievement 
data for any assessment. The simple 
difference in these measures between 
two assessments of the same exercise 
(or sets of exercises) provides six 
measures of' change in achievement 
that are routinely used in National 
Assessments change reports. The next 
section describes how standard errors 
are estimated for the 12 statistics 
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used in NAEP reports; 

Computation of 
Standard Errors 

> 

In order to obtain an approximate 
►measure of the sampling vaiiability 
in the statistics (1) through (6) , c a 
jackknife replication procedure for 
estimating the sampling variance of 
nonlinear statistics from complex, 
multistage samples was tailored to 
National Assessment's sample design. 
Miller (1968, 1974) and Mosteller and 
Tukey (1977) provide *. inforratipn 
about the jackknife technique, while 
Folsom (1977) describes how the pro- 
cedure is used in estimating standard 
errors for National Assessment's sam- 
ple design. 

To demonstrate the computational as- 
pects of this technique, consider 
estimating the variance of -the sta- 
tistic^, in- (1)-- the proportion of 

age-el igibles in subgroup _i who would 
select response category j. on exer- 
cise e. 

This -statistic is based on the data 
from all the n^ replicates in the H 

strata. Let P?2 hk be defined as a 

replication estimate of ?V and 

constructed from 'all the replicates 
excluding the data from replicate k 
in stratum h. These replication esti- 
mates are computed, as if the excluded 
replicate had not responded * and a 
reasonable nonresponse adjustment is 
used to replace the. data Jn„xeplicate 
hk in estimating Pp. Several choices 

for replacing the data in replicate 
hk are available. , In order to obtain 
a convenient and computationally ef- 
ficient algorithm for approximat- 



ing standard errors, National Assess- 
ment replaces C^ k and W? hk from the 
hkth replicate with correspdnding 

sums from another paired replicate in 
the same stratum. The replicate esti- 
mate is then computed. The replicate 
estimates to be used in the caXcular 
tions are determined by arranging all 
the • replicates in each stratum into 
successive pairs. That is, replicate 
1 is paired with replicate 2, repli- 
cate 2 with replicate 3, 3 with 
4,...(n h -l) with n^ and replicate n^ 
-**• ™ * ■ * *— 

with replicate 1. , 



The contribution to the variance of 

P^ 5 by each pair of replicates is the 

change in tfte value of the statistic 
incurred by replacing the data from 
- each replicate in the pair with the 
data from the other replicate in the 
pair and recomputing P®3 in the 

usual way. This produces two repli- 
cate estimates. Squaring the differ- 
ence between these replicate esti- 
mates and then ^dividing by eight 

measures the contribution of this 
pair of replicates to the total vari- 
ance. The sum of these contributions 
over all n^ successive pairs in the 

stratum is the contribution by stra- 
tum h to the total variance. Ihe 
square root of the sum of the H stra- 
tum contributions is thg-estimate of 
the standard error of P. . 
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Algebraically, the two replicate es- 
timates for the pair k,' k+1 ,£ where 
k=l » • • -Ik and ^+1=1) are: 



(7) P' 



and 



ej 



(8) P 



ej 



c e ^ - 

i++ . 


L ihk L ih(k+1) 


u eJ - 

i-H- 


W e * + W ej 
W ihk W ih(k+l) 


■c e3 


- c ei +c eJ 
C ih(k+1) U ihk 


s» ■ 

w i++ 


u e J + W ej 
" W ih(k+1) ihk 



The contribution to the total vari- 
ance from stratum h is: 

(9) var (P e i) Z h f? ei - P eJ V . 

K } K ill 8 £ ^ r i-hk r i-h(k+l)/ 

And finally, an estimate of the stan- 
dard error of P e J is: 

i 



(10) SE (P< j ) ' (I var P^) . 
1 * h 



Multiplying by 100, yields the 

percentage of response to category 2. 

Multiplying SE(P e;] ) by . ioo yields 
i 

the corresponding estimated standard 
error of the percentage. 



In general, the jackknifed standard 
errors of the proportion estimates 
will be larger than the simple random 
sampling formula (PQ/N) 1/l § 



* • • 

where P=P?^, Q=l-P and N is the num- 
i 

ber of sampled respondents in sub- 
group _i who took the exercise. The 
larger size of SE(P^) reflects 

mainly the loss of precision due to 

cluster-sampling of schools and stu- 
dents. The .standard errors' for the 
achievement measures (2) throuqh v (6^ 
are computed through a serie^s Pof 
steps analogous to those followed in 

computAng S"(P?->) .* 



The standard errors for the differ- 
ences between two assessments for any 
of the achievement measures CI) 
through (6) are computed as the 
square root of the sum of. the squared 
standard errors from each of the sep- 
arate assessments. 

The size of the standard errors de- 
pends largely r^ot only on the number 
of replicates and schools included in 
the sample, but also on the number of 
'respondents in each of the 1 reporting 
groups. Table I>1 shows the average 
number of students* responding to an 
exercise booklet for each of the re- 
portinq qroups for each age for each 
assessment year. Table D-2 shows 
National Assessment's current esti- 
mates of the proportions of students 
in each reporting group at each age. 
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TABLE D-L. Average 
• .* Ah Item 



Number of Respondents in Reporting Grbups Taking 
Booklet , by Age and Assessment Year+ 



Nation ♦ 
*■ 

Begion 
Northeast 
Southeast 
Central 
West m 

.Sex - * 
Male 
Female 

• , 
Race/ethnicity 

White * S 

Black 

Other-hf 

Parental education 

• Not graduated high scIkxdI 
Graduated high school » 
Post hiah school 
Unkncwn++ \ 

Type of coTtWunity 
Rural 

Disadvantaged urban 
Advantaged urban 
Other +4- 

Size of community t 
Big cities 

Fringes around big cities 
Medium cities 
Small places 

Grade 
, 3, 7, K) 
4, 2, 11 
- 12 

Other-H- . 



i Age 9 

1971-72 1978^79 



" Age 13 1 
1971-72 1978-79 



2,538 

4 




— 2,584 


2,762 


2,317 


627' 


580 . H 


- 629 , 


675 


563 


631 


6?5 


658 


657 


" 595 


653 


665 


658 


* 752 


589 


626 


639 

- 


638 


681 

0 


569 


1,255 


1,255 


1,297 


1,370 


1,126 


1,283 


1,255 


1,286 


1,395 


1,191 


£,977 


1,849 


2,069 


✓ 

2,053 


1,890 


387 


484 


340 


507 


288 


174 


177 

* * 


175 


202 


i39 


y 282 


191 


'36 


326 


468 


607 


589 ' 4 


789 


895 


715 


840 


830 


998 


1,200 


1,006 


809 


850 


361 

- i 


341 


128 


252 


252 


258 


271 


233 


253 


250 


261 


281 


232 


1 253 


253 


256 


277 


235 


1,780 


1,755 


1,809 


1,933 


1,617 


484^. 


723 


510 


• i 
' .775 


435 


526 


448 


561 


588. 


507 


343 


237 


348 


287 


303 


1,184 


M.,097 


v 1,164 


1,115 


1,071 


6£4 


623 


667 


683* 


280 


1,807 


1,334 


1,799 


2,031 


1,681 










255 


87 


53 . 


"nd 1 


48 


101 



- A 

Age 17* 
1971-72 1978-79; 



642 
683 
725 
649 



1,312 
1,386 



2,134 

392. 
176 



386 
895 
1,334 
87 



246 
306 
265 
1,885 



718 
571 
287 
1,123 



360 
2,014 
285 
4T 



+Data may not total due to rounding error. 
* #Seventeen-yrar-olds enrolled in school, 
-KData are not reported for these groups. 



TABLE D-2. Estimated Current Population Proportions 
' of National Assessment Reporting Groups 
for In-School Students 



Reporting Groups 



Age 9 



Age 13 



Age 17 



Sex 
Male 
»Female, 



.50 
.50 



.50 
.50 



.48 

,52 



Race/ethnicity 
, 'White 

Black " x 

Other " 



.79 
.14 
.07 



.80 
.13 
.07 



,83 
.12 
,05 



Reg ion 

Northeast 
Southeast 
Central 
West 



.25 
.22 
.27 
.26 



25 
23 
27 
25 



25 
20 
29 
26 



Parental education 

Not graduated high school 
, Graduated high school 

Post high ^chool 

Unknown 



,09 
,24 
.33 
.34 



13 
32 
42 
, 13 



.15 
.32 
.48 
.05 



Type of community . 
Rural 

Disadvantaged urban 
'Advantaged urban 
Other 



,08 
,07 
,11 
,74 



.10 
.07 
.11 
.72 



. 08 
.09 
.11 
.72 



Size of community ■ . 

Big cities .20 

Fringes around big cities" .22 

Medium cities ^ .12 

Small places^ " .46 



.21 
.22 
. 11 

.46 = 



.19 
.26 
,11 
.44 



Grade 

>.<3, 
3", 

' 4, 
>4, 



in 

<7, 
7, 
8, 

>8, 



schpol 
<10 
10 

11 

12 



Other 



<.01 
.23 
.75 
<.01 
<.01 



.02 
.25 
.72 
<.01 
<.01 



.02 
.13 
.75 
. 10 
<.01 
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, APPENDIX E 

ADJUSTMENT OF RESPONDENT WEIGHTS BY SMOOTHING .TO REDUCE 
RANDOM VARIABILITY OF ESTIMATED POPULATION PROPORTIONS 



Background 

As noted elsewhere, a weight 
is assdgned to every' individu- 
al who responds to an exercise 
administered in an assessment. 
The weight is the reciprocal 
of the probability of selec-' 
tion of the individual with 
adjustment for monresponse, 
and the probabilities of se- 
lection are based on the esti- 
mated number of people in the 
target age populations. There- 
fore, the weight for an indi- 
vidual estimates the number of 
people that* the individual 
represents in the age popula- 
tion. 1 The surrT of the weights 
of all individuals at an age 
level who responded to an ex- 
ercise is an estimate of the 
total number of people in that 
age population in the year 
that the exercise was as- 
sessed^ Similarly, the sum of 
weights for all individuals 
who /took the exercise and who 
also are members of some 'demo- 
graphic category (such as 
blacks) gives an estimate of 
the number of. people £n thd 
age population, for the year, 
who are also members of the 
category. The ratio of ^the two 
totals estimates <he p.jpor- 
tional representation of the 
demographic category in the 
age population for the given 
year. 



Separate estimates of the pro- 
portional representation of 
the various demographic sub- 
groups are provided by each 
booklet administered to a' 
particular age group in a 
given year. Due to random sam- 
pling variability, the esti-^ 
mates of population propor- 
tions for a given year based 
on single booklets will vary. 
There is also random sampling 
variation in estimates of pop- 
ulation proportions from- year / 
to year in addition to what- 
ever trends in population pro- 
portions over time that might 
.•exist. - , 

It is desirable to reduce the 
random variability of popula- 
tion proportions as much as 
possible since this variabil- 
ity has an effect on perform- 
ance estimates. For example, 
the* peYceht-age of acceptable 
responses -for an age group is 9 
a function of the relative 
proportions of high-performing 
and low-performing groups, ft 
the/ relative proportions of 
these groups are very differ- 
ent, in different assessments 
due to sampling variability, 
then a portion, of the change 
in percentage c f ^ acceptable- 
responses for an age group 
could be attributable to^ 
yearly sampling differences in , 
the relative proportions of-- 
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high- and low-achieving 

groups . 

In addition to reporting per- 
formance estimates for an age 
group as a whole, National 
Assessment also reports per- 
formance for various 
subpopulations , such as whites 
or blacks * Because var iabil ity 
of .sub-groups within these ^jb- 
populations (such as males and 
"females 'within the white sub- 
population) influences the 
performance estimates for tie 
subpopulations, it is desira- 
ble that fluctuations of pro- 
portions of all subgroups of 
each subpopulat ion be reduced 
as much as possible. 

For each age and year, each of 
the various booklets 
administered provides 
estimates of a given 
population proportion. Since 
these estimates are subject to 
book! et- to-book 1 et 
variability, a better estimate 
of the population proportion, 
which will' have reduced 
variability,^ is obtained by 
combining tHe information Srom 
all booklets. However, these 
proportions vary from yfear to 
year due to random sampling 
variability or systematic 
differences ii^ sampling 
procedures. An even better 
estimate of .population 
proportions for any single 
year can be obtained by 
smoothing the proportions over 
several assessment years . The 
word "smoothing" is. used here 
in the sense of fitting a 
smooth curve to a sequence of 
numbers by robust /resistant 
procedures (Tukey, 1977). 
Smoothing estimates of 
population proportidns reduces 
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a large portion of the 
sampling variability while 
preserving, as far as 
possible, actual' trends 
occurring in the age 
population. 

After the population pro^or- \ 
tions hav,e been smoothed, sd- x 
justed weights are derived for v 
the assessed individuals so \ 
that the population propor- 
tions computed us I ng the ad- 
justed weights are equal to 
the smoothed proportions. The 
adjusted weights are then used x 
for all analyses. 

Smoothing Procedures Used by 
National Assessment 

The most direct way to smooth 
proportions is first to clas- 
sify people into mutually ex- 
clusive multiway cells on the 
basis of their membership in 
categories of various impor- - 
tant var iables and then to 
smooth the proportions within 
each of the resulting multiway 
cells across years. Unfortu- 
nately, this procedure tends 
to produce a large number of 
C-11.S with few people and, 
consequently, quite unstabl'e 
estimates of smoothed propor- 
tions. 

To circumvent this difficulty, 
National Assessment has uti- 
lized various smoothing proce- 
dures. These procedures , wh ich 
are all basically weighting- 
class adjustments ap>p] ied 
independently to each age, are 
designed to control, to var- 
ying deg rees , f 1 uctua tions in 
certain key subgroups while 
'avoid ing , ,as much as possible, 
instabilities^ due to smal] 
cells . 
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The procedure used in 197^-79 
has the following characte is- 
tics: 

1. It produces a single ad- 
justed • weight for each 
individual. 

2. It affords good control on 
-the distribution of pro- 
portions of certain key 
variables. 

3. It tends to produce sta- 
bility of performance es- 
timates, j 

i 

4. It is relatively easy to 
implement. 

i 

Even though adjusted weights, 
using this procedure differ/ 
slightly from the correspond- 
ing adjusted waights from the 
other procedures that have 
previously been employed, Na- 
tional Assessment intends to 
use weights obtained using the 1 
1978-79 procedure for all fu- 
ture analyses of data assessed 
in earlier years. This is sim- 
ply because \ NAEP believes 
weights obtained through this 
procedure to be the best 
available . 



The Current Smoothing 
Procedure 

The first step in the 1978-79 
smoothing procedure involved 
the partitioning of the popu- 
lation of age-class eligibles 
into the six smoothing cells 
given in Table' E-l,. The same 
cells were used for,*all ages. 



Then, for 
year, the 
population 
cells was 
given age 
portion of 
pa rt icular 
as the sum 



each age and every 
proportion of the 
in each of the 
estimated . For a 
and year, the pro- 
the population in a 
cell was computed 
of weights of all 



respondents assessed in the 
g i ven year who were of the 
spec i f i ed age arid who belonged 
_in the cell, divided by the 
total of the weight of all 
respondents of the given age 
assessed in that year. 

Each of the six cells was com- 
prised of a sequence of esti- 
mated population propo rt ions 
corresponding to the various 
years of assessment . Each such 
sequence of proportions was 
then smoothed by fitting 
robust/resistant lines. Using 
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Cell 

1 

2 
3 
4 
5 
6 



STABLE E-l. Smoothing Cells Used for the 
1978-79 Smoothing Procedure 



Race 

White 
White 
White 
Black 
Black 
i Other 



Regior^ 

All \ 
All 
All 

Southeast 
Not Southeast 
All 
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Community Size 

Big city + fringe 

Medium city 

Small places 

All 

All 

All 
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data from the U.S. Census and 
Current * Population Survey r 
trends in enrollment by age 
and race and by age and region 
were obtained. The data from 
these surveys were adjusted to 
correspond with National As- 
sessment definitions as much 
as possible* The resistant 
lines within-, the smoothing 
cells were constrained to sat- 
isfy the trend from the U.S. 
Census • and Current Population 
Survey data. 

The final step in the smooth- 
ing procedure was to adjust 
the respondents 1 weights to be 
consistent with the smoothed 
proportions. Since each re- 
spondent takes only one book- 
let, the weight adjustments 
were done independently for 



each booklet. For a given age, 
year and bookl et, population 
proportions using the original 
weights were obtained for each 
of the smoothing cells. Then 
the weights of all respondents 
of a qiven cell were multi- 
plied by the ratio of the 
smoothed eel 1 propo rt ion to 
the proportion using the orig- 
inal weights. This produced 
the adjusted weights that were 
used in all analyses. 



Adjustment of Weights 
1 by Users 

The smoothed population pro- 
portions for 9-, 13- and 17- 
year-olds (in-school only) are 
given 1 *r. Tables E-2, E-3 and 
E-4, r -o^cctively . The columns 



TABLE E-2. Smoothed Frequencies From 10-Year Smooth 





by Smoothing Cell 


and Year 


for 9-Year- 


-Olds 










Cell 








1 


2 


3 


4 


5 


6 


Race 


White 


White 


White 


Black 


Black 


Other 


Region 


All 


All 


All 


SE+ 


Not SE 


All 


Size of 














Community 


BC+FR# 


MC++ 


SP## 


All 


All 


All 


Year 














1969-70 


0.3293 


0. 1258 


0. 3546 


0.0535 


0. 0745 


0.0624 


1970-71 


0. 3251 


0. 1224 


0. 3601 


0.0547 


0. 0746 


0.0631 


1971-72 


0. 3210 


0. .1191 


0. 3656 


0. 0558 


0. 0747 


0.0639 


1972-73 


0. 3168 


0. 1158 


0. 3711 


0.0570 


0.0748 


0. 0646 


1973-74 


0. 3126 


0. 1124 


0. 3766 


0. 0581 


0.0748 


0.0654 


1974 -75 


0. 3085 


0. 1091 


0. 3822 


0.0592 


0.0,19 


0.0661 


1975-76 


0.3043 


0. 1058 


0. 3877 


0.0604 • 


0. 0750 


0.0668 


1976-77 


0.3001 


0.1024 


0. 3932 


0.0615 


0.0751 


0.0676 


1977-78 


0. 29 59 


0.0991 


0. 3987 


0.0627 


0. 0752 


0. 0683 


1978-79 


0.2918 


0. 0958 


0.4 04 2 


0. 0638 


0.0753 


0.0691 



+SE = Southeast. 

#BC+FR = Big Cities + Fringes. 

++MC = Medium Cities. 

##SP = Small Places. 
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TABLE E-3. Smoothed Frequencies From 10-Year Smooth 
by Smoothing Cell and Year for 13-Year-01ds 



Cell 



Race 
Reg ion 
Size of 
Community 

Year 



1 

White 
All 



2 

Wh ite 
All 



BC+FR# MC++ 



3 

White 
All 

SP## 



4 

Black 
SE+ 

All 



5 

Black 
Not SE 

All 



+SE = Southeast. 

jfBC+FR = Big Cities + Fringes. 

++MC = Medium Cities. 

#|SP = Small Places. 



6 

Other 
All 

All 



1969- 


70 


0. 


3200 


0. 


1309 


0. 


3703 


0. 


0513 


0. 


0673 


0. 


0602 


1970- 


71 


0. 


3201 


0. 


1256 


0. 


3731 


0. 


0521 


0. 


0681 


0. 


0610 


1971- 


72 


0. 


3202 


0. 


1202 


. 0. 


3760 


0. 


0528 


0. 


0690 


0. 


0618 


1972- 


73 


0. 


32~3 


0. 


1149 


0. 


3788 


0. 


0536 


0. 


0698 


0. 


06 26 


1973- 


74 


0. 


320 3 


0. 


1096 


0. 


3816 


0. 


0544 


0. 


0714 


0. 


0642 


1974- 


75 


0. 


3205 


0. 


1043 


0. 


384 5 


0. 


0552 


0. 


0714 


o; 


0642 


1975- 


76 


0. 


3206 


0. 


0989 


0. 


3873 


0. 


0560 


0. 


0722 


0. 


0650 


1976- 


77 


0. 


3207 


0. 


0936 


0. 


3901 


0. 


0567 


0. 


0731 


0. 


0658 


1977- 


78 


0. 


3207 


0. 


0883 


0. 


3929 


0. 


0575 


0. 


0739 


0. 


0666 


1978- 


■79 


0. 


3208 




0830 


0. 


3958 


0. 


0583 


0. 


0747 


0. 


0674 



of each table represent the 
smoothing cells while the rows 
represent the assessment year. 
For example, the smoothed pop- 
ulation proportion of 9-year- 
olds in smoothing cell 2 
(whites in medium cities) for 
1972-73 is .1158. 

To adjust respondent weights 
to be consistent wi th the 
smoothed proportions, the fol- 
lowing procedures were fol- 
lowed : 

1. For each booklet, respon- 
dents were classified ac- 
cording to smoothing cell, 
and the raw population 
proportions for each cell 
were obtained. For exam- 



ple, the raw proportions 
for a booklet given to 
9 -year-olds in smoothing 
cell 4 was the total of 
the weights of all 9- 
year-olds receiving the 
booklet who were black and 
in the Southeastern re- 
gion, divided by the total 
of the we ights of all 
respondents receiving the 
booklet. 



For each booklet and 
smoothing cell, a weight 
adjustment factor as the 
rat io of the smoothed pop- 
ulation proportion (for 
the appropr iate age , year 
and smoothing cell) over 
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TABLE E-4. Smoothed Frequencies From 10-Year Smooth 
by Smoothing Cell and Year for 17-Year-01ds 

Cell 





1 


2 


3 


4 


5 


6 


Race 


White 


White 


White 


Black 


Black 


Other 


Reg ion 


All 


All 


All 


SE+ 


Not SE 


All 


Size of 














Community 


BC+FR# 


MC++ 


SP## 


All 


All 


All 


Year 














1969-7C 


0.3405 


0. 1447 


0.3686 


0.0415 


0.0581 


0.046* 


1970-7.1 


0.3419 


0. 1386 


0. 3704 


0. 0427 


0.0591 


0.0472 


1971-72 


0.3432 


0. 1326 


0. 3722 


0. 0440 


0. 0602 


0. 0478 


1972-73 


0.3446 


0.1265 


0. 3740 


0.0452 


0.0612 


0.0484 


1973-74 


0. 3460 


0.1204 


0.3759 


0. 0465 


0.0622 


0.0491 


1974-75 


0. 3474 


0.1143 


•0.3777 


0.0477 


0.0633 


0. 0497 


1975-76 


0.3487 


0. 1082 


0. 3795 


0. 0490 


0.0643 


0.0503 


1976-77 


0.3501 


0.1021 


0. 3813 


0.0502 


0.0653 


0.0509 


1977-78 


0.3515 


0. 0961 


0. 3831 


0.0515 


0.0664 


0.0515 


1978-79 


0.3529 


0.0900 


0. 3849 


0. 0527 


0.0674 


0.0522 



+SE = Southeast. 

#BC+FR = Big Cities + Fringes, 

++MC = Medium Cities. 

##SP = Small Places. 



the raw population pro- 
portion was obtained. 

3. The adjusted weights for 
an individual were the 
product of that individu- 
als original weight and 
the appropriate adjustment 
facto r . 

Changes in Smoothed 
Proportions as 1 New 
Assessments Are Completed 



Every time an 
completed , a new 
addefi to each of 
of population 



assessment is 
time point is 
the sequences 
proportions 



within the smoothing cells. 
.This means that, even though 
robust/resistant procedure, 
are used, the addition of a 
new point may somewhat change 



the values of smoothed propor- 
tions for prior years. Addi- 
tionally, any changes in meth- 
odology will impact the esti- 
mates. 

Th is means that the smoothed 
proportions, with the addition 
of the next assessment data, 
are apt to differ somewhat 
from the corresponding 

smoothed proportions without 
the new data. National Assess- 
ment has adopted the philos- 
ophy that the smoothed propor- 
tions, based on all currently 
available data using the best 
available algorithm, are the 
best available. Therefore, all 
subsequent analyses, for any 
year, will be done using uhis 
best available information, 
even though this may produce 
estimates that slightly differ 
from prior values. 
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GLOSSARY OF NATIONAL ASSESSMENT TERMS 



Acceptable response . lAny re- 
sponse to an exercise that 
demonstrates achievement /of 
the objective measured by 
that exercise. 



Administration time . The 
total time allowed on the 
paced audio tape for an 
exercise. (Includes the 
time allowed for the stimu- 
lus and the response.) 



Administ rat ion 



timetable . 



Time periods during the 
school year when the vari- 
ous age groups are as- 
sessed. The time periods 
are : 



October-December 
January-February 
March-May 



13-year-olds 
9-year-olds 
17-year-olds 



Age group or age level . Three 
age groups have been sam- 
pled in both music assess- 
ments : 9-year-olds , 13- 
year-olds and 17-year-olds 
attending school. Birth 
date ranges for each age 
group in each of the two 
assessments are as follows: 



Assessment Age 9 Age 13 Age 17 
1971-72 1962 1958 10/54-9/55 
1978-79 1969 1965 10/6^-9/62 



Assessment . The documentation 
of the progress . in 'knowl- 
edge, skills and attitudes 



of American youth. Measures 
are taken at periodic in- 
tervals for each learning 
area, with the goal of de- 
termining trends and 



termining trends ana re- 
porting the findings to the 
public and to the education 
community . 



Assessment 



administrate r . 



Individual employed to 
administer the assessment 
in par tic ipa ting schools. 

Backg round questions . Ques- 
tions about respondents 1 
instructional experiences 
with music in school and 
out of school were included 
in music item booklets . 
Stand a rd background ques- 
tions asked in every learn- 
ing a rea are found on the 
back pages of the item 
" booklets and include such 
things as level of parental 
education and reading ma- 
terials in the home . Back- 
ground questions used in 
the 1978-79 music assess- 
ment appear in Appendix B. 

Booklet . Items (exercises) 
are presented to respon- 
dents^ in booklets. Booklets 
are designed to be scored 
by optical .scanning ma- 
chines. Each booklet con- 
tains (1J instructions for 
answering" items and sample 
items , (2 ) assessment items 
and (3) background ques- 
tions. Each booklet con- 
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tains approximately 30-35 
minutes of assessment items 
and 10-15 minutes of intro- 
ductory material and back- 
ground questions. 

Category (scoring) . A classi- 
fication of a response to 
an open-ended exercise. See 
scoring guide . 

Category within a variable . A 
subclassi f i cat ion wi thin a 
variable. For example, male 
and female are categories 
of the variable sex. See 
reporting groups . 

Difficulty level . The per- 
centage of acceptable re- 
sponses to an exercise. 

Exercise . A task designed to 
measure an objective. Be- 
cause NAEP does not admini- 
ster "tests," but instead 
describes educational 
achievement over time, the 
term "exercise" is often 
used instead of the term 
"item" or "test item." The 
terms "it^m" and "exercise" 
are used synonymously in 
this report. 

Exercise booklet . See book- 
let. 



Exercise part . See item part . 

Exercise pool . The entire set 
of exercises prepared for a 
learning area. This set 
includes recycled, exer- 
cises, exercises developed 
f o r previous assessments 
but not used due to exer- 
cise booklet or budgetary 
constraints and newly de- 
veloped exercises. 



Field test . A pretest of ex- 
ercises to obtain informa- 
tion regarding clarity, 
difficulty levels, timing, 
feasibility and special 
administrative problems 
needed for revision and 
selection of exercises to 
be used in the assessment. 

Grade in school . Results are 
reported for 9-year-olds in 
the 3rd and c 4th grades; 
13-year-olds in the 7th and 
8th grades; and 17-year- 
olds in the 10th, 11th and 
12th grades. 

Group administration . Book- 
lets were administered to 
groups of 10 to 25 students 
in 1978-79. In 1971-72/ 
group size was 12 respon- 
dents. A paced audio tape 
was^used to provide uniform 
instructions and oral pres- 
entation of exercises. 

Hand scoring (scoring) . The 
coding of responses in a 
format compatible with the 
optical scanning equipment 
being used. Multiple- 
choice exercises can be 
directly machine scored; 
however, responses to 
open-ended exercises must 
be coded in scoring ovals 
so that they can then be 
machine scored. See scor ing 
guide . 

ID number . An ident i f icat ion 
number referring to the 
un ique number assigned to 
each respondent. This num- 
ber is assigned to preserve 
the anonymity of each re- 
spondent. NAEP does not 
keep records of the names 
of any individuals. 
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Item* See exercise . 

Item booklet . See booklet . 

Item part . Each part of an 
item ' that asks a separate 
question. Parts may all 
pertain £o one stimulus, 
such as a graph or a table, 
or may concern the same 
topic. 

Jackknife . The name of the 
algorithm used by NAEP to 
estimate standard errors of 
percentages and other sta- 
tistics. 

Learning area . One of the 
areas assessed by National 
Assessment : read lng/1 i ter- 
ature, writing, mathemat- 
ics, science, citizen- 
ship/social studies, art, 
music, career development. 
Also called "subject area." 

• Level of parental education . 
These levels are described 
in Appendix A, 

Modal grade , The p grade in 
which the majority of each 
in-school age group is en- 
rolled. For 9-year-olds, 
the medal grade is the 4th 
grade; for 13-year-olds, 
the 8th grade; and for 17- 
year-olds, the 11th grade. 

Objective , A desirable educa- 
tion goal agreed upon by 
scholars in the field, edu- 
cators and concerned lay 
persons, and established 
through , the consensus ap- 
proach. 

Objectives redevelopment . 

After the initial assess- 
ment of a learning area, 



one of the first steps in 
prepar ing for reassessment 
is a revi ew o f the 
learning-area objectives. 
This is carried out by 
scholars in the field, edu- 
cators and concerned lay 
persons. These reviews may 
result in revision, modifi- 
cation or total rewriting 
of the learning-area objec- 
tives to reflect current 
curricular goals and empha- 
ses; they may also result 
in the endorsement of the 
objectives from the previ- 
ous assessment^as adequate 
for the next assessment. 

Open-ended exercise . A 

nonmul ti pie-choice exercise 
that requires some type of 
written or oral response. 

Paced audio tape . A tape re- 
cording that accompanies 
each booklet to ensure uni- 
formity in administration. 
Instructions and exercises 
are playe,d back from the 
tape recording so thc<t 
reading di f f icul t ies^ will 
not- interfere with an indi- 
vidual's ability to re- 
spond , Response time is 
included on the tape. 

Primary sampling unit (PSU) . 
First-stage sampling units, 
typically a county or a 
group of contiguous coun- 
ties. 

Principal ' s questionnaire . A 
data collection form given 
to school officials, The 
officials respond to ques- 
tions concerning enroll- 
ments, s ize of the commu- 
nity, occupational composi- 
tion of the community, and 



69 



76 



so forth. Samples of these 
questionnaires are found in 
Appendix B. See also sup- 
plementary pr incipal ; s 

quest ionna ire . 

PSU. See primary sampling 
unit . 

Public-use data tapes . Com- 
puter tapes containing 
respondent- level e exercise 
and backg round/demog raphic 
data and machine-readable 
documentation. These tapes 
a*:e available for use by 
external researchers wish- 
ing to do secondary analy- 
ses of National Assessment 
data . 

Racial/ethnic category . For 
the music assessments, re- 
sults are reported for 
whites, blacks and His- 
panos. 

Receipt control . Procedures 
implemented by scoring 
staff to check in and 
screen materials from the 
field. Information gained 
from receipt control proce- 
dures is relayed to assess- 
ment administrative staff 
so that any errors may be 
corrected . 

Re eye led exercises . The set 
of exercises that is kept 
secure from one assessment 
to the next to be used to 
measure changes (growth, 
stabil i ty or decline) in 
performance for the^ learn- 
ing area. 

Region . One of four geograph- 
ical regions used in ga- 
thering and reporting 3ata: 
Northeast , Southeast, Cen- 



tral and West. States in- 
cluded in each region are 
shown in Appendix A. 

Released exercise . An exer- 
cise for which results and 
exerc ise text have been 
reported to the public. 

Released exercise set . A set 
of released exercises, ip- 
cluding documentation and 
sco ring guides, that can be 
purchased from National 
Assessment. Data for the 
released exerc ise set are 
included as addendum pages. 

Reporting groups . Categories 
of variables for which Na- 
tional Assessment data are 
reported . Variable cate- 
gories are defined in Ap- 
pendix A. 

Rescore . If an open-ended 
exerc ise was scored under 
d if f erent cond i t ions than 
presently held or if pas- 
sage of time might affect 
scoring, responses from a 
previous assessment may be 
re sco red at the same t ime 
that responses from a later 
assessment are scored . Re- 
sponses from an earlier 
^assessment also may be held 
and not scored so that they 
can be scored with respons- 
es from a later assessment. 

Respondent . A person who re- 
sponds to the exercises in 
an assessment booklet . 



Response options . Different 
alternatives to a 

multiple-choice question 
that can be selected by the 
respondent. 
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Review conference . A con- 
ference held to review the 
objectives of a learning 
area to assure their 
acceptance as measures of 
the objectives by scholars, 
educators and < lay persons 
or to review exercises for 
racial, ethnic^ social or 
regional -bias. 

Sample . National Assessment 
does not assess an entire 
age population but rather 
selects a representative 
sample from the age group 
to answer assessment items. 
(See Chapter 4 for a de- 
scription of National As- 
sessment sampling procedur- 
es . ) 

Scoring guide . A guide for 
hand scoring an open-ended 
exercise that specifies 
descriptive or diagnostic 
categories by giving defin- 
itions and sample respons- 
es . 

Scoring ovals . Scannable 
ovals printed beside 

multiple-choice options and 
printed at the bottom of 
the page for open-ended 
exercises (to be used in 
hand scoring) . When ovals 
are marked, they can be 
scored by machine and re- 
sponses recorded by com- 
puter. 

Sex . Results are reported for 
males and .females. 

Size of community . Results 
are reported for four 
size -of -commun i ty cate- 
gories: big cities, fringes 
around big cities, medium 
cities and small places. 



These categories are de- 
fined in Appendix A. 

SMSA . Standard Metropolitan 
Statistical Area. SMSAs are 
economic and social units 
defined by the U.S. Bureau 
of the Census. 

Standard error . A measure of 
sampling variability for a 
statistic. Because of 

NAEP's complex sample de- 
sign, standard errors are 
estimated by jackknifing 
first-stage s'ample esti- 
mates. 

Stem . The portion of ap exer- 
cise that states the prob- 
lem or asks the question. 

Stimulus . Fo r music exer- 
cises , this may be an aural 
or a visual stimulus used 
part of the stem. 

Subject area . See learning 
area . 

Subpopulation or subgroup . 
Groups within the national 
population, such as males 
and females, for which data 
are reported . 

Supplementary pr incipal 1 s 

quest ionna ire . A data col- 
lection form given to 
school officials. On thfs 
form, officials are asked 
to respond to questions 
concerning course offer- 
ings, materials and staf- 
fing specific to the learn- 
ing area being assessed . A 
sample of this question- 
naire is found in Appendix B. 
See also principal's ques- 
tionnaire. 
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Tape script . A script prepared 
for the announcer to use in 
producing the paced tape. 
It indicates exactly v*nat 
is to be read or not read 
aloud to the students and 
indicates the 1 amount of 
response time allowed, for 
each .exercise,. See paced 
audio tape . 

s 

Timing . Most NkEP exercises 
are administered with a 
paced audio tape to stan- 
dardize data col lection 
conditions*. The , tape in- 
cludes the amount of time 
students a re a 1 lowed to 
respond to each exercise. 

Type of community . Results 
are reported for three 
type-of -community cate- 
gor ies : disadvantaged 
urban, advantaged urban and 
rural. Definitions of these 
categories are found in 
Appendix A. 



User r tape . See public-use 
data tape . 

Variable . A classification of 
respondents. Standard re- 
porting variables are: re- 
gion, sex, race/^fchnici ty , 
level of parental * educa- 
tion, size of community, 
type of community and grade 
in school. 

We iqht . A jnult ipl icative fac- 
to r equal to the reciprocal 
of the probabil i ty of a 
respondent being selected 
for assessment with adjust- 
ment for nonre^ponse — .an 
estimate of the number of 
persons in the population 
represented by a respondent 
in the sample. Theoreti- 
cally, the sum .of weights 
for all respondents at an 
age level is equal to <the 
number of persons in the 
country at that age level. 
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